assess-knowledge

Assess Knowledge

Check what the user knows about a specific topic area and update the knowledge state.

ARGUMENTS

Receives a topic area to assess (e.g., "prompt injection", "AWS IAM", "SIEM/SOAR").

Process

Step 1: Read existing knowledge state

Use Read tool on .research-kit/profile/knowledge.md. Look for entries related to the topic.

• If the topic has a [solid] entry: inform the calling agent that user is solid on this. No need to ask questions.
• If the topic has a [learning] entry: ask 1-2 quick questions to see if level has improved.
• If the topic has a [gap] entry or no entry exists: proceed to Step 2.

Step 2: Quick assessment (only if needed)

Ask the user 2-3 short questions to gauge their level. Questions should be:

• Practical, not theoretical trivia
• Progressively harder: basic concept -> application -> edge case
• Answerable in 1-2 sentences

Example for "AWS IAM":

1. "What's the difference between an IAM role and an IAM user?"
2. "How would you restrict an EC2 instance to only access a specific S3 bucket?"
3. "What's the risk of using wildcard (*) in resource ARNs for cross-account access?"

Based on answers:

• All correct with confidence -> [solid]
• Basic correct, advanced unclear -> [learning]
• Struggles with basics -> [gap]

Step 3: Update knowledge state

Use the Edit tool to update .research-kit/profile/knowledge.md:

• Find the relevant domain section (e.g., ## Cloud Security)
• If domain doesn't exist, create it
• Add or update the entry with the assessed level

Format: - [level] topic description

Step 4: Return assessment

Report back:

• Topic assessed
• Level determined (solid / learning / gap)
• Brief reasoning

Important

• Keep assessment quick -- max 3 questions, don't turn this into an exam
• If user says "I don't know anything about X", accept that as [gap] without quizzing
• If user says "I know this well", accept as [solid] unless the calling agent has reason to doubt
• Always be respectful of user's self-assessment