BAEM1N.DEV
RSS Feed
A space for sharing hands-on experiments in AI, data engineering, and
automation. Covering RAG pipelines, AI Agents, LLMOps, and practical
benchmarks.
Written by Bae Gimin (baem1n), Founder of DDOK.AI and an AI/Data
Educator & Engineer. Benchmark posts prioritize reproducible
methodology, public source code, raw CSV/results, and clear failure
conditions. Contact:
gm.bae@ddok.ai
.
Trust & disclosure: when the author maintains a related open-source
project, that relationship is disclosed in the article. Experimental
claims are tied to code, raw data, or measurement notes whenever
possible.
Subscribe via
RSS
for updates.
Recent Posts
176B on a Single 128GB Box — Serving Qwen3.8-Flash-Next on GB10, Down to the CUDA Kernel Every official quant overflowed memory. Building an unmerged PR got an 87 GiB 4-bit model running — then CUDA died at 260K tokens. I traced it to the grid.y limit of 65,535, reproduced it in two lines without the model, and fixed it.
Building PhoenixCallbackHandler: Wrapping OpenInference Tracing as a LangChain Callback A package design for a Phoenix LangChain CallbackHandler that reuses OpenInference's tracer instead of reimplementing run-to-span conversion.
Tracing LangChain with Arize Phoenix: auto_instrument vs CallbackHandler A deep dive into Phoenix's official register(auto_instrument=True) path, how OpenInference instruments LangChain, and when a callback handler API is useful.
Getting Started with Arize Phoenix: Open-Source LLMOps for Tracing, Evaluation, and Debugging A practical introduction to Arize Phoenix, how it compares with LangSmith and Langfuse, and why OpenTelemetry/OpenInference matter for LLM observability.
Stacking Univariate Winners Didn't Give the Optimum — A 384-Combination Korean RAG Sweep Scoring all 384 Pre×Retrieval×Reranker combinations for Korean RAG — query2doc, only 4th by univariate e2e judge, becomes the global winner (judge 4.067/acc 0.827) once paired with jina-reranker-m0. The MRR winner ≠ the answer-quality winner. Interaction is why the full sweep was needed.
How Far Have Open-Weight LLMs Come in Korean RAG — 46 Generators and Judge Reliability Comparing 46 Korean RAG generators (27 open + 19 closed) — gpt-oss-120b and kimi-k2.5 tie for the open-weight lead (acc 0.740), and gpt-oss-20b reaches 0.727 at 13GB VRAM. The closed leader gpt-5.4 (0.787) is -4.7pp ahead. A single LLM-as-Judge shook the rankings.
Why a 0.6B Korean Reranker Beat a 4B SOTA — Comparing 25 Rerankers for Korean RAG Univariate comparison of 25 rerankers for Korean RAG — a 0.6B Korean fine-tune (dragonkue/bge-reranker-v2-m3-ko) hits MRR 0.7697, beating the 6.7× larger 2025 SOTA Qwen3-Reranker-4B (0.7514) by +1.83pp. The reranker was the single biggest axis.
Dense Alone Wasn't Enough: BM25-KIWI, Hybrid, and Query Transforms in Korean RAG Univariate retrieval comparison for Korean RAG — Hybrid 3:7 (Dense + BM25-KIWI) hits MRR 0.7171, beating every single-method retriever. BM25 needs morphology (KIWI): +14.4pp over whitespace. Pre-retrieval query transforms were noise-level on their own.
Korean RAG Ingestion: Simpler Choices Won in Loader, Chunker, Embedding Univariate comparison over 300 Korean Q&A — PyMuPDF wins the loader at MRR 0.6486; the top char chunker by dense MRR is Chonkie Fast 800 (0.6903), but within ≈1.5pp noise, so the standard LC Recursive 300/50 (0.6816 dense / 0.7171 hybrid) was adopted downstream; KoE5 beats an 8B English model by +0.16 MRR. Korean alignment mattered more than processing complexity.
Korean RAG Benchmark: Why I Took the Whole Pipeline Apart with 300 Questions Methodology of a Korean RAG benchmark that decomposes the pipeline into 6 stages and runs a full 384-combination Cartesian sweep. 300 Q&A × 58 PDF × 5 domains, 46 generators (27 open + 19 closed), 4-metric LLM-as-Judge, ≈1.2M LLM calls.