Retinal ganglion cells (RGCs) form an array of feature detectors, which convey visual information to central brain regions. Characterizing RGC diversity is required to understand the logic of the underlying functional segregation. Using single-cell transcriptomics, we systematically classified RGCs in adult and larval zebrafish, thereby identifying marker genes for >30 mature types and several developmental intermediates. We used this dataset to engineer transgenic driver lines, enabling specific experimental access to a subset of RGC types. Expression of one or few transcription factors often predicts dendrite morphologies and axonal projections to specific tectal layers and extratectal targets. In vivo calcium imaging revealed that molecularly defined RGCs exhibit specific functional tuning. Finally, chemogenetic ablation of eomesa+ RGCs, which comprise melanopsin-expressing types with projections to a small subset of central targets, selectively impaired phototaxis. Together, our study establishes a framework for systematically studying the functional architecture of the visual system.