Write your own challenges, in your own repository.

Why a CLI

Real challenges are codebases, not form fields. They deserve the tools you already use for code, an editor, Git, code review, CI.

From install to published, then inspectable

Everything up to Publish works offline and without an account.

# installs the `codepraxis` command
pip install codepraxis
codepraxis --version

codepraxis --example

Build a SQL Agent
  Medium · Python, SQL · 60 minutes

  https://…azurecontainerapps.io/?folder=/home/praxis/querybot

codepraxis new my_challenge
codepraxis validate --local my_challenge

my_challenge  (local)
  solution  2/2 passed
  starter   0/2 passed
  PASSED  0.3s

codepraxis --login

Logged in as you@company.com
Publishing as: Acme Corp

codepraxis --publish my_challenge

Validating my_challenge in the runner before publishing…
  solution  2/2 passed · starter 0/2 passed · PASSED

About to publish my_challenge to Acme Corp, as a draft.
Continue? [y/N] y

my_challenge draft for Acme Corp
  challenge 123, version 456
  https://…azurecontainerapps.io/?folder=/home/praxis/my_challenge

codepraxis --list
codepraxis --edit 123 --title "Debug webhook delivery" --status draft
codepraxis --edit 123 --open

What a challenge is made of

A directory the platform mounts into a container. The candidate gets source/; the grader runs ._tests/test_1.py.

my_challenge/
├── metadata.json            # {"name": "my_challenge"}, becomes the workspace folder
├── backend.conf             # {"BACKEND": "AI", "LANGUAGE": "PYTHON"}
├── setup.sh                 # optional; installs dependencies
├── source/                  # what the candidate starts from
│   ├── main.py
│   └── README.md            # current state + how to run it
├── ._tests/
│   └── test_1.py            # a `testCases` class
└── ._course_data/
    ├── course_toc.json      # selects the active test file
    └── feature.md           # the Instructions tab
solution/                    # SIBLING of the pack, never uploaded

class testCases:
    def __init__(self, user_wxpace) -> None:
        self.RUN = 2                    # cases shown to the candidate
        self.RunCaseInputs = ["1 2 3", "-5 5"]
        self.userWxpace = user_wxpace   # the workspace path
        self.default_timeout_window = 20000   # MILLISECONDS
        self.msg = ""

    def test_case_1(self, timeout_window=20000, override=1):
        ...
        self.msg = "PASS"              # this decides pass/fail
        return expected_text, actual_output

Five rules the runner enforces. codepraxis lint checks all of them without executing anything.

• One constructor argument. The runner calls testCases(workspace_path). A second parameter breaks the pack.
• self.msg == "PASS" decides the verdict. The return value is what the candidate panel displays, not whether the case passed.
• No zero-padded names. Ordering uses the first number in the method name, so test_case_01 and test_case_1 collide.
• RUN is the visible count. Extra test_case_* methods beyond it still run as hidden cases. RunCaseInputs must have RUN entries.
• Timeouts are milliseconds. Not seconds.

Two kinds of check, three verdicts

One reads your files. The other runs them. You will use both, at different moments.

validate runs the static checks first and stops if they find an error, executing a pack whose grader cannot even be constructed only produces a confusing traceback.

Local checks are honest about their limits: they do not run setup.sh, do not have the container image's packages, and run on your OS rather than linux/amd64. Anything they cannot verify is reported as a note. Publishing always requires --remote.

What happens when you publish

Publishing is deliberately strict, because a published challenge can be put in front of a candidate immediately.

• A full remote validation runs first. Local results never qualify. Reuse an earlier passing run with --validation-run-id if you have one.
• A reference solution is required. It is what proves the challenge is solvable at all.
• Ownership comes from your API key. The CLI never sends a company, the platform derives it, so a misconfigured client cannot publish into someone else's catalog.
• It publishes as a draft. Unless you pass --live.
• A preview URL is returned when available. The platform starts a container from the published pack so you can inspect the question immediately.
• The uploaded artifact excludes your solution. Only the pack directory is packaged; the reference answer never leaves your machine except for validation.

# No prompts: token from the environment, --yes to skip confirmation
export CODEPRAXIS_TOKEN="sk_live_…"

codepraxis lint
codepraxis --publish my_challenge --yes

List and edit existing questions

The CLI can also work with questions that already exist for the company attached to your API key. Listing is read-only; editing updates metadata and opens a container preview.

codepraxis --list

123    Debug webhook delivery    draft      v456
124    Build a SQL Agent         published  v457

codepraxis --edit 123 --title "Debug webhook delivery" --difficulty 2

Updated question 123: Debug webhook delivery
  https://…azurecontainerapps.io/?folder=/home/praxis/debug_webhook_delivery

Metadata edits include --title, --description, --status, --difficulty, --max-time, --max-attempt, --tech-stack, and --ai-enabled / --no-ai-enabled. Use --open to launch the returned container URL in your browser.

Changing the starter code, the tests or the brief means re-publishing the pack. Pass the question’s id, or you get a second copy of it.

codepraxis --publish my_challenge --challenge-id 123

my_challenge updated for Acme Corp (draft)
  challenge (new version of) 123, version 458

With --challenge-id the platform validates as usual and then adds a new version to question 123, keeping its id, its assignments and its history. Without it, publish always creates a new question, correct the first time, a duplicate every time after. Versions are additive, so candidates already mid-assessment keep the version they started on.

codepraxis --delete 123

Permanently delete question 123? This cannot be undone.
Continue? [y/N]

Deletion is refused once the question has been assigned to anyone, attempts and reports still point at it, and removing it would orphan work candidates and reviewers depend on. To take an assigned question out of circulation, put it back in draft instead:

codepraxis --edit 123 --status draft

Authoring with Claude Code

If you use Claude Code, one command teaches it the pack format so you are not explaining the contract every time.

codepraxis --install claude-plugin

It writes a local plugin into your project and prints the two commands to enable it. You get:

• /codepraxis:new , design the question architecture, implement the pack, validate locally, publish through the remote runner, and return the preview URL.
• /codepraxis:validate , validate and fix what fails, in a loop.
• A pack-authoring skill , loads automatically when Claude touches a pack, so it already knows the testCases contract and the two-fixture rule.

Commands

--root points any pack command at a different directory. lint, validate, --publish and --list also take --json for machine-readable output, always one document, however many packs ran, and -v to show passing cases and the checks a tier could not verify.

Environment

Getting unstuck