Module 3 Lesson 4: Model Sizes and Variants
Understanding the trade-offs of scale. Why a 70B model is smarter than an 8B model, and why you might not want to use it.
Understanding the trade-offs of scale. Why a 70B model is smarter than an 8B model, and why you might not want to use it.
Talking to the machine. Why prompting a local 8B model requires a different approach than ChatGPT.
Words as they happen. Why streaming is the secret to a fast-feeling AI application.
Put your knowledge to the test. Compare Llama, Mistral, and Gemma on speed, humor, and logic.
The engine under the hood. A non-math guide to the Transformer architecture that powers all modern LLMs.
Compressing intelligence. How we fit 100GB models into 5GB files without making them stupid.
The universal file type. Why GGUF is the 'PDF of AI' and why it's the foundation of the Ollama ecosystem.
How much can the AI remember? Understanding the relationship between context windows and RAM usage.
The bridge between words and numbers. How LLMs translate your typing into something a computer can process.
Optimization 101. Balancing speed vs quality vs memory in your local AI setup.
Hands-on: Benchmarking your machine. Compare quantization levels and measure memory usage in real-time.
The blueprint of a model. Understanding how to configure your AI using simple text files.
Mastering the commands. A deep dive into FROM, SYSTEM, PARAMETER, and ADAPTER.
The power of instruction. How to write effective system prompts that transform your model's personality.
Fine-tuning the engine. A dictionary of PARAMETER options to control speed, creativity, and memory.
Standing on the shoulders of giants. How to create layers of custom models using the FROM command.
Creating stable AI systems. How to ensure your custom models remain the same over time.
Hands-on: Creating a specialized AI persona from scratch. Move beyond the default registry.
The universe of open AI. Understanding the scale of Hugging Face and how it relates to Ollama.
Know your rights. A guide to AI licenses (MIT, Apache, Llama) and what they mean for your business.
Not all models are equal. Understanding which architectures (Llama, Mistral, BERT) work with the Ollama engine.
The DIY path. How to take a raw PyTorch model and turn it into a GGUF file for Ollama.
Going deep on compression. Exploring the technical differences between Q4_0, Q4_K_M, and GQA.
Is it working? How to verify that your imported Hugging Face model is behaving correctly in Ollama.
Hands-on: The full workflow from Hugging Face download to Ollama creation.
How Ollama handles memory. Understanding why the 'second' run is always faster than the 'first'.
Managing the gigabytes. How to clear space and move your Ollama model library to a larger drive.
Squeezing every drop of performance. How to force Ollama to use the GPU and manage shared memory.
Stability over scope. Why lowering your context window can actually make your AI feel faster and more stable.
Processing at scale. How to optimize Ollama for high-volume tasks like document digestion.
Hands-on: Benchmarking your machine. Compare quantization levels and measure memory usage in real-time.
The universal bridge. How to talk to Ollama from any programming language using HTTP requests.
Words as they happen. How to handle NDJSON streams in your application for a professional AI feel.
The AI Engineer's standard. Using the official Ollama Python library to build smart scripts.
AI in the browser and the server. Building with the Ollama JavaScript library.
Building complex AI workflows. connecting Ollama to the world's most popular AI orchestration framework.
Giving the AI hands. How to let local models run functions, check the weather, or query a database.
Hands-on: Creating a fully functional, streaming terminal chatbot using Python and Ollama.
Optimization for 8B. Why 'Chain of Thought' is the secret weapon for making small models act like giants.
Hardening the persona. Using system prompts as a defensive layer to prevent 'Jailbreaking' and off-topic conversations.
Precision generation. Techniques to limit the model's verbosity and ensure it stays within character limits.
AI that speaks code. How to force Ollama to output valid JSON every single time.
Stick to the facts. Techniques to prevent local AI from making up information.
Hands-on: Combine system prompts, JSON mode, and negative constraints to build a production-ready data extractor.
The Grand Finale. Apply everything from Modules 1-7 to build a fully automated system that cleans raw data, performs analysis, and chooses the best AI model.
An absolute beginner's guide to understanding programming and why Python is the perfect first language.
Save time and frustration by learning how to identify and fix the most common Python errors made by beginners.
Apply everything you've learned in Module 1 by building four practical Python projects from scratch.
A step-by-step guide to installing Python and setting up your coding environment correctly.
Learn how to create, save, and execute your first Python script using VS Code.