On the Uniform Convergence of Consistent Confidence Measures
Explore this paper's citation graph
Summary
It is shown that finite VC-dimension is sufficient for guaranteeing the consistency of confidence measures produced by empirically consistent classifiers and implies that one can calibrate confidence measuresproduced by any existing algorithms with monotonic functions, and still get the same generalization guarantee on consistency.
- Type
- preprint
- Published
- 2015-06-09
- Cited by
- 0
- References
- 23
- Access
- Open access
- OpenAlex
- https://openalex.org/W593154044
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:14834163
Keywords
Interpretability, Consistency (knowledge bases), Measure (data warehouse), Confidence interval, Computer science
References
- NESTOR: A Computer-Based Medical Diagnostic Aid That Integrates Causal and Probabilistic Knowledge.
- Pattern Recognition and Machine Learning
- Assessing the Calibration of Naive Bayes Posterior Estimates
- Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers
- Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
- On the Density of Families of Sets
- Beyond myopic inference in big data pipelines
- CrowdER: Crowdsourcing Entity Resolution
- A Uniform Convergence Bound for the Area Under the ROC Curve
- Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications
- Predicting good probabilities with supervised learning
- Measures of Distributional Similarity
- Minimizing Uncertainty in Pipelines
- Knows what it knows: a framework for self-aware learning
- Blackwell Approachability and No-Regret Learning are Equivalent
- Scikit-learn: Machine Learning in Python
- On the Consistency of Multiclass Classification Methods
- Understanding Machine Learning From Theory to Algorithms 1st Edition Shwartz Solutions Manual
- Rademacher and Gaussian Complexities: Risk Bounds and Structural Results
- Sparseness Versus Estimating Conditional Probabilities: Some Asymptotic Results
Cited by
No citing papers recorded for this paper.
Related papers
- On the Statistical Consistency of Plug-in Classifiers for Non-decomposable Performance Measures
- Cost-Sensitive Learning with Noisy Labels
- Confidence Sets with Expected Sizes for Multiclass Classification
- Least Ambiguous Set-Valued Classifiers With Bounded Error Levels
- Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPs
- Not All are Made Equal: Consistency of Weighted Averaging Estimators Under Active Learning
- Guaranteed Validity for Empirical Approaches to Adaptive Data Analysis
- Bayes Consistency vs. H-Consistency: The Interplay between Surrogate Loss Functions and the Scoring Function Class
- A general framework for label-efficient online evaluation with asymptotic guarantees