What the job looks like day to day
The title covers a wide range, from classic prediction models to work on large language models. Common tasks:
- Preparing data: collecting, cleaning, labeling and splitting it so the test set stays honest.
- Training and fine-tuning: choosing a baseline, training, and adapting pretrained models to a task.
- Evaluating: measuring quality on held-out data, slicing results by user group and tracking regressions.
- Serving and scaling: packaging models behind APIs, managing GPU memory, batching and latency.
- Monitoring: watching for drift, errors and cost after launch, then retraining.
- Deciding build vs buy: comparing a hosted API with an open-weight model you run yourself. Our guide to open-weight vs closed models covers the trade-offs.
Google's Rules of Machine Learning are a good picture of the mindset: Rule #1 is "Don't be afraid to launch a product without machine learning," because a simple heuristic often beats a model with too little data.
Skills table
| Skill | Why it matters | How to show it |
|---|---|---|
| Python and core libraries | Almost all ML work runs in Python | A clean repository with tests and a README |
| Statistics and evaluation | Bad splits and leaky test sets ruin results | An evaluation report with held-out data and error slices |
| Deep learning frameworks | Training and fine-tuning need PyTorch or similar | A fine-tuned model with training logs |
| Working with pretrained models | Most new projects start from an existing model | A Hugging Face model card you wrote |
| Serving and infrastructure | Models must run fast and affordably in production | A deployed endpoint with latency and cost per 1,000 requests |
| Hardware sizing | Memory decides which models you can run | A note on why a model fits your GPU, with numbers |
| Communication | Stakeholders need limits stated plainly | A one-page summary of what the model cannot do |
Hardware sizing is now a daily skill. OpenAI's gpt-oss-120b model card states the 117B-parameter model with 5.1B active parameters fits a single 80GB GPU, while gpt-oss-20b runs within 16GB of memory. Explaining figures like these is part of the job.
Learning path with free official resources
- Foundations. Google's Machine Learning Crash Course offers animated videos, interactive visualizations and hands-on practice exercises covering core concepts.
- Deep learning by doing. fast.ai's Practical Deep Learning is described as "a free course designed for people with some coding experience," with lessons on deployment, neural net foundations, NLP and data ethics.
- Large language models. The Hugging Face LLM course is free and without ads; each chapter is designed for about a week at 6 to 8 hours.
- Engineering judgment. Read Google's Rules of Machine Learning end to end before your first production project.
- Evaluation. Learn Inspect, the open-source evaluation framework from the UK AI Security Institute, and follow our AI evaluation learning path.
- Context and cost. Our guide to context windows explains why long inputs lose recall and how length tiers change price.
The Hugging Face LLM course, checked 17 October 2026, is free and designs each chapter for about one week at 6 to 8 hours of work.
Pay: which BLS occupation applies
The US Bureau of Labor Statistics does not have a machine learning engineer occupation. Three occupations cover most of the work:
- Computer and information research scientists. The BLS page for computer and information research scientists gives a median annual wage of $140,300 in May 2025, projects 22% employment growth from 2025 to 2035, and says these workers typically need a master's or higher degree in computer science or a related field.
- Software developers. ML engineers on product teams often sit here. The BLS software developers page lists a $135,980 median in May 2025.
- Data scientists. Modeling-heavy analytics roles fit the data scientists occupation at $120,230 in May 2025, per the BLS data scientists page.
BLS reports a $140,300 median annual wage for computer and information research scientists in May 2025, with 22% projected growth from 2025 to 2035.
These are national medians for broad occupations, not ML engineer offers at specific companies.
Portfolio ideas
- End-to-end classifier. Pick a public dataset, train a baseline and a better model, evaluate on held-out data, deploy behind an API and report latency.
- Fine-tune a small open model on a narrow task, publish a model card with training data, metrics and limits.
- API vs self-hosted cost study. Run the same task on a hosted API and an open-weight model on rented hardware, and report cost per 1,000 tasks and quality.
- Retrieval system with an evaluation. Build search over a document set and measure answer accuracy on 50 questions, before and after an improvement.
- Drift monitor. Simulate data drift and show your monitoring catching it.
Each project should run from a fresh clone with one command. Related roles: the prompt engineer guide for application work, and our careers hub for the full list.
How to get your first role
Entry routes into ML engineering usually run through an adjacent job:
- From software engineering: take on data pipelines, model serving or evaluation tooling for an existing ML team.
- From data science or analytics: move one of your models into production and own its monitoring.
- From research or a graduate program: publish code and model cards alongside papers so employers can see engineering quality.
Employers screen for evidence that a model you built ran reliably for real users. Show logs, latency, cost and what broke, not only accuracy on a benchmark.
How we researched this
We read Google's Machine Learning Crash Course and Rules of Machine Learning, fast.ai's course page, the Hugging Face LLM course, the Inspect documentation, OpenAI's gpt-oss-120b model card, and the BLS pages for computer and information research scientists, software developers and data scientists on 17 October 2026. Pay figures are national medians for related occupations.
Frequently asked questions
Is machine learning engineer a BLS occupation? No. BLS reports pay for related occupations such as computer and information research scientists, software developers and data scientists.
How much do machine learning engineers make? BLS does not publish the title. Computer and information research scientists had a $140,300 median annual wage and software developers $135,980 in May 2025.
Do I need a master's degree? For research-heavy roles often yes; BLS says computer and information research scientists typically need a master's or higher. Product-focused ML engineering roles often hire from software development with a bachelor's.
What is the best free course to start machine learning? Google's Machine Learning Crash Course for concepts, then fast.ai's Practical Deep Learning for building. Add the Hugging Face LLM course for language models.
What is the difference between a data scientist and an ML engineer? Data scientists focus on analysis and modeling for decisions; ML engineers focus on building and running models in production. Many roles mix both.
Do ML engineers still train models from scratch? Rarely for language tasks. Most projects fine-tune or prompt a pretrained model and spend more effort on data, evaluation and serving.
What hardware do I need to learn ML? A laptop with access to a cloud notebook service is enough to start the courses above. Larger open models need a GPU; gpt-oss-20b, for example, runs within 16GB of memory per its model card.