Professional & research work
A&P Consulting
Self-hosted language-model infrastructure and document-grounded application development, from an API proof of concept to Flux.
- My role
- Junior Developer
- Context
- Professional work · A&P Consulting · 2025–Present
- Toolkit
- Qwen · vLLM · Python · RunPod · pgvector
- My contribution
- Self-hosted model serving and document retrieval, from a small API proof of concept to Flux integration and validation.
- Evidence shown
- Local-validation screenshots and dated acceptance notes. The Flux walkthrough is simulated; production operation is not implied.
Self-hosted model serving
The initial proof of concept explored a self-hostable language model exposed through an OpenAI-compatible API. It kept model selection, remote serving, a Python client, and evaluation small enough to validate independently.
In my Junior Developer work, the progression connected Qwen model serving on vLLM/RunPod with authenticated requests, configurable prompts, token budgets, error behavior, and repeatable benchmark runs. The August handoff reports Tickets 1–7 complete and the temporary GPU infrastructure intentionally terminated afterward.
- Python CLI
- Authenticated OpenAI-compatible API
- vLLM on RunPod
- Qwen model response
What the benchmark does—and does not—show
The small sequential suite reported mean response latency of 0.704 seconds without thinking and 9.846 seconds with thinking. Its limited reasoning subset reported 3/3 with sufficient thinking budget versus 1/3 baseline.
These are dated, small-sample observations, not general accuracy estimates, throughput measurements, or service guarantees. Model recommendation scores are also distinct from measured benchmark results.
Flux: document retrieval and cited answers
Flux V2 walkthrough
Flux is a separate application repository that builds on the serving work. Its document-grounded workflow connects PDF validation and extraction, overlapping chunks, embeddings, pgvector retrieval, and an answer interface that can return document sources.
The documented local stack combines a Next.js frontend, FastAPI, PostgreSQL/pgvector, and separate chat/embedding model services. Scoped issues and acceptance checks keep an upload endpoint, retrieval behavior, and provider integration from being treated as one untestable milestone.
October validation notes report a successful grounded browser answer and automated live acceptance. A longer streamed response still failed, and cancellation had not been manually verified. That boundary matters: a successful basic answer did not finish the full interaction.
- Validated PDF
- Extract + chunk + embed
- pgvector retrieval
- Grounded answer + sources
Integration lessons from local acceptance
The notes record a tuple/list adapter mismatch at the pgvector boundary and a later invalid-streaming-response error despite an initial HTTP 200. A route responding successfully is not sufficient evidence that progressive generation completes.
The POC’s 9B serving setup and Flux’s later 27B local-validation service are separate configurations. Production hosting and authentication work should not be inferred from frontend preparation or a local test.
Reliability appears at the boundaries
This work connects infrastructure and application behavior: authentication, model settings, data adapters, retrieval, and streaming all change what a user actually experiences.
The strongest lesson is to make each claim match its test. An API proof of concept, a grounded response, and a production-ready application are different achievements.