Systematic measurement biases make normalization an essential step in single-cell RNA sequencing (scRNA-seq) analysis. There may be multiple competing considerations behind the assessment of normalization performance, of which some may be study specific. We have developed "scone"- a flexible framework for assessing performance based on a comprehensive panel of data-driven metrics. Through graphical summaries and quantitative reports, scone summarizes trade-offs and ranks large numbers of normalization methods by panel performance. The method is implemented in the open-source Bioconductor R software package scone. We show that top-performing normalization methods lead to better agreement with independent validation data for a collection of scRNA-seq datasets. scone can be downloaded at http://bioconductor.org/packages/scone/. Keywords: RNA-seq; methods; normalization; preprocessing; quality control; scRNA-seq; single-cell.