DraftPilot

Overview

DraftPilot started from a simple frustration: writing tailored resumes and cover letters is time-consuming, and most tools either oversimplify it or generate things you can’t fully trust.

The idea was to build something structured, where you can take a job description, analyze it, and generate drafts that are actually usable and editable.

What made it difficult

One big issue with LLMs is that they can sound confident while being wrong. In a multi-step system, those small mistakes get carried forward and become harder to catch.

Formatting was another challenge. Especially with LaTeX, things have to be exact. Close enough isn’t good enough.

Approach

I designed it as a pipeline: parse → analyze → score → generate → review.

The backend is in FastAPI, with a React frontend so each step is visible. Instead of relying on a single prompt, I broke the process into stages where each part can be inspected and adjusted.

Generation is constrained using templates to keep things consistent and reduce drift.

Iteration and debugging

Early versions looked good at first glance but had issues like repeated phrasing or subtle inaccuracies.

I added logging between steps, tightened constraints, and limited where free-form generation was allowed. In some cases, I replaced open-ended generation with more structured outputs.

Evaluation

Instead of a single metric, I evaluated based on:

• formatting correctness
• alignment with actual user input
• overall clarity and tone

The focus was on making sure the system is reliable, not just fluent.

Takeaways

The biggest lesson here was that structure matters a lot. Breaking things into clear, inspectable steps made the system much more trustworthy.

It also showed me that “agents” are only useful if you can actually see and control what they’re doing.