About Oleg
I help companies design and operationalize reliable evaluation and validation systems for LLM-driven workflows and AI applications.
My focus is on reproducibility, longitudinal comparability, traceability, and governance-ready reporting for evolving AI systems operating in production or regulated environments.
Typical focus areas:
- LLM evaluation frameworks
- model and prompt drift monitoring
- validation and revalidation workflows
- audit-ready evaluation reporting
- dataset/version management
- workflow reliability and operational traceability
- production Python evaluation pipelines
Typical deliverables include evaluation methodology design, baseline dataset creation with expected-behavior annotations, scoring and aggregation systems, reproducible evaluation runs, and operational validation processes tied to model, prompt, tooling, or workflow changes.
I prioritize pragmatic architectures that preserve longitudinal comparability and operational continuity while avoiding unnecessary rewrites.
My work is especially relevant for AI systems where reliability, governance, auditability, or operational accountability matter.
English
Native or bilingual
Remote only
Primarily works remotely
Experience
- Independent Consultant (IndrasNet/DLX initiative)AI Systems Architect | LLM Evaluation & ReliabilitySOFTWARE PUBLISHINGMarch 2022 - Today (4 years and 3 months)Nuremberg, GermanyI build evaluation, validation, and reliability systems for LLM-driven workflows where outputs must remain reproducible, comparable over time, and defensible under audit or risk review.My work focuses on longitudinal comparability across model/prompt/tooling changes, evaluator reliability, versioned ground truth, traceability, governance-ready reporting, and drift/revalidation processes for evolving AI systems.Typical delivery areas include evaluation methodology design, production Python evaluation pipelines, baseline dataset creation, audit-ready reporting, and operational validation processes for AI systems deployed in regulated or high-assurance environments.
- Digital Montenegro GmbHCEODIGITAL AND ITJune 2018 - February 2022 (3 years and 8 months)Bar, MontenegroLed digital business operations and strategic initiatives with focus on structured execution, operational oversight, and scalable process coordination across cross-functional activities.Responsibilities included:
- strategic planning and operational management
- financial oversight, budgeting, and performance monitoring
- coordination of partners and external stakeholders
- process structuring for scalable delivery workflows
- governance-oriented operational practices across evolving digital environments
This experience strengthened the operational and governance-oriented approach I now apply to AI/LLM evaluation and validation systems, including reproducibility, traceability, and reliability under changing conditions.
Recommendations
Be the first to recommend Oleg
Help this freelancer shine by sharing your experience working together.
These freelancer profiles also match your criteria
Agatha Frydrych
Backend Java Software Engineer
4.7
(3)
2
Baptiste Duhen
Fullstack developer
4.6
(4)
5
Amed Hamou
Senior Lead Developer
4
(2)
7
Audrey Champion
Web developer
4.3
(3)
4
Education
- Master’s Degree in FinanceOdessa State Economics University1998Studied Finance and Credit with specialization in public budgeting and securities. The program covered financial systems, capital markets, state budget management, and financial analysis.
- Postgraduate Studies in Financial ManagementOdessa State University2000Focused on business administration and financial management, including organizational finance, operational planning, and financial decision processes.