Sounds like it's working as it's supposed to.
The file formats that are not selected in Enable are just greyed out instead of hidden.
(don't remeber about very early versions of mac os x)
Maybe your thinking of photoshop 6 and 7 on mac os 9, where the open dialog had Show instead of Enable
where the file formats were hidden other than the one selected next to Show.
Or photoshop cs2/cs3 that had the alternate Adobe Dialog that did the same thing