How to Write Science Questions that Are Easy for People and Hard for Computers
Explore this paper's citation graph
Summary
This work argues that hand-constructed multiple-choice tests, with questions that relate the formal science to the realia of laboratory experiments or of real-world observations are likely to be easy for people and hard for AI programs.
- Type
- article
- Published
- 2016-04-13
- Cited by
- 21
- References
- 26
- Access
- Open access
- OpenAlex
- https://openalex.org/W2734841799
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:26496686
Keywords
Realia, Computer science, Simple (philosophy), Mathematics education, Test (biology)
References
- Verbal IQ of a Four-Year Old Achieved by an AI System
- Never-Ending Learning
- A study of the knowledge base requirements for passing an elementary science test
- A Question-Answering System for AP Chemistry: Assessing KR&R Technologies
- The initial knowledge state of college physics students
- Probase: a probabilistic taxonomy for text understanding
- Chemistry: The Central Science
- CYC: Using Common Sense Knowledge to Overcome Brittleness and Knowledge Acquisition Bottlenecks
- The scope and limits of simulation in automated reasoning
- Solving Geometry Problems: Combining Text and Diagram Interpretation
- Chemistry: The Central Science, 10th Edition
- CAPTCHA: Using Hard AI Problems for Security
- Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
- ConceptNet 3 : a Flexible , Multilingual Semantic Network for Common Sense Knowledge
- A Beautiful Mind The Life Of Mathematical Genius And Nobel Laureate John Nash
- The Winograd Schema Challenge
Cited by
- Toward a Comprehension Challenge, Using Crowdsourcing as a Tool
- The Measure of All Minds: Evaluating Natural and Artificial Intelligence
- Solving Mathematical Puzzles: A Challenging Competition for AI
- Deep Learning: A Critical Appraisal
- University Entrance Exam as a Guiding Test for Artificial Intelligence
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- The Meta-Turing Test
- Advances in Automatically Solving the ENEM
- Does it Make Sense? And Why? A Pilot Study for Sense Making and Explanation
- CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
- What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge
- SemEval-2020 Task 4: Commonsense Validation and Explanation
- ECNU-SenseMaker at SemEval-2020 Task 4: Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanation
- Is this sentence valid? An Arabic Dataset for Commonsense Validation
- The Winograd Schemas from Hell
- Benchmarks for Automated Commonsense Reasoning: A Survey
- A Novel Psychometrics-Based Approach to Developing Professional Competency Benchmark for Large Language Models
- Concept and Level of Carbon Price, and Ex-post Evaluation of Carbon Pricing Policy
- A Benchmark Arabic Dataset for Commonsense Explanation
- General-Purpose Question-Answering with Macaw
Related papers
- The Use of Realia in Teaching Speaking
- "Some Historical Realia in Russian and Romanian: Lexical and Phraseological Correspondences"
- Realia Found in Certain Spanish Textbooks Used in Massachusetts' High Schools
- Realia Found in Certain Spanish Textbooks Used in Massachusetts' High Schools
- Miniature objects as representations of realia
- Functions of Words Denoting Cultural Realia in Pelevin’s “Prince of Gosplan” and its English Translation
- Transadapting the Cultural Realia of Soviet Russia to an Arab Audience: The Case of the Film Утомленные Солнцем (Burnt by the Sun)
- THE WAYS OF TRANSLATING REALIA IN THE NOVEL “DIVERGENT” BY V. ROTH
- Лингвистический статус термина реалия