Research Paper Reviewer
Conduct comprehensive, critical reviews of research papers targeting top-tier computer science conferences by autonomously analyzing content, identifying issues, and providing constructive feedback.
When to Use This Skill
- Reviewing research paper drafts before conference submission
- Conducting peer-review-style analysis of academic papers
- Identifying logical gaps, unclear statements, and missing evidence
- Analyzing paper structure and flow across sections
- Preparing papers for conferences like OSDI, NSDI, SOSP, SIGCOMM
- Getting critical feedback before actual submission
Core Principles
Apply these principles when reviewing papers:
- Clarity and Precision: Prioritize clear, unambiguous, and precise language for technical audiences
- Fluency: Ensure text flows naturally and reads smoothly
- Appropriate Vocabulary: Use terminology common in technical and systems research papers
- Logical Cohesion: Critically assess logical flow and identify weak links in arguments
- LaTeX Integrity: Respect original LaTeX syntax - only modify textual content within commands/environments when making edits
Writing Guidelines to Enforce
When reviewing, check for these writing standards:
Hyphen Usage
- Avoid hyphens for connecting independent clauses
- Bad: "The system is fast - it processes data quickly"
- Good: "The system is fast, processing data quickly"
- Exception: Compound adjectives (e.g., "state-of-the-art") are acceptable
Voice Preference
- Prefer active voice for directness
- Preferred: "We implemented the prototype"
- Avoid: "The prototype was implemented by us"
- Passive voice is acceptable when the object is more important than the actor
Tense Guidelines
- Present tense for authors' work: "We implement a prototype..."
- Past tense for previous literature: "Smith et al. proposed..."
Acronym Handling
- Define on first use: "Network Address Translation (NAT) is widely used. NAT helps..."
- Check for undefined acronyms
Conciseness
- Eliminate redundancy without sacrificing clarity
- Pages are limited - be cautious about suggesting expansions unless critical
Audience Context
- Target readers: Graduate students, professors, researchers in computer science
- Writing style: Normal for this audience, not oversimplified
- Page limits: Very limited, so avoid suggesting expansions unless critical
Review Approach
Act as a critical but constructive peer reviewer, focusing on six key areas:
1. Weaknesses in Arguments
- Identify logical gaps or weak reasoning
- Question methodology and assumptions
- Challenge claims lacking evidence
- Note unsupported generalizations
2. Unclear Statements
- Flag ambiguous or confusing sentences
- Identify jargon without definition
- Note inconsistent terminology
- Highlight unclear references
3. Missing Evidence
- Point out claims needing substantiation
- Request benchmark results or comparisons
- Identify missing experimental validation
- Note absent ablation studies or sensitivity analyses
4. Alternative Interpretations
- Present counterarguments
- Suggest alternative explanations for results
- Question generalizability of findings
- Challenge assumptions
5. Novelty and Significance
- Assess clarity of contributions
- Question novelty over prior work
- Evaluate practical impact and applicability
- Check for clear differentiation from related work
6. Reader Confusion
- Identify aspects that might confuse readers
- Note structural issues
- Flag inconsistencies between sections
- Check for missing context or background
Output Format
Frame feedback as reviewer questions or constructive criticisms:
- "A reviewer might ask: How does this approach compare to X in terms of overhead?"
- "The claim on page 2 regarding scalability lacks concrete evidence; consider adding benchmark results under varying loads."
- "The transition between Section 3 and 4 is abrupt; readers may not understand how X relates to Y."
Reference Format
Always include file paths and line numbers when referencing issues for easy navigation:
- Format:
paper.tex:145
- Example: "The undefined acronym 'DPDK' appears in
introduction.tex:23"
Review Workflow
Follow this systematic workflow:
1. Read Paper Files
Use the Read tool to load all relevant paper files (.tex, .bib, etc.)
2. Search for Patterns
Use Grep to identify common issues:
- Undefined acronyms
- Inconsistent terminology
- Overuse of passive voice
- Missing citations
- TODO markers or comments
3. Analyze Structure
- Check logical flow across sections
- Verify appropriate section organization
- Assess transitions between sections
- Check for missing or redundant content
4. Identify Issues
Apply the six review areas above systematically across all sections
5. Provide Consolidated Feedback
Organize feedback by:
- Section or file
- Severity (critical, major, minor)
- Type (clarity, evidence, logic, etc.)
6. Suggest or Make Edits (Optional)
Only make edits if explicitly requested by the user
Important Constraints
- Provide no advice when no meaningful improvement can be made
- Avoid pedantic or nit-picking comments
- Aim for conference acceptance, not perfection
- Only suggest significant improvements that impact acceptance likelihood
- Be self-consistent with severity assessments across similar issues
- Do not make edits unless explicitly requested
Target Audience for Reviews
Research papers targeting top-tier computer science conferences:
- Systems: OSDI, NSDI, SOSP, EuroSys, ATC
- Networking: SIGCOMM, NSDI, CoNEXT
- Security: Oakland, USENIX Security, CCS, NDSS
- Other top-tier CS conferences