AI Lab 10: Handwritten Digit Recognition with AI
Lab Overview
In this lab, you will build a complete, end-to-end deep learning application that recognizes handwritten digits. You will train a Convolutional Neural Network (CNN) on the classic MNIST dataset, evaluate how well it performs, and then connect the trained model to a live webcam so it can recognize digits you write on paper in real time.
The entire workflow runs on Windows 11 inside a Jupyter Notebook, using a dedicated conda environment and VS Code as the editor. No GPU is required — the model is small enough to train on the CPU in a few minutes.
| Item | Detail |
| Goal | Train a CNN on MNIST and perform real-time digit recognition from a camera |
| Platform | Windows 11 + Miniconda + VS Code + Jupyter Notebook |
| Framework | TensorFlow / Keras (CPU) |
| Estimated time | 3–4 hours (across environment setup, training, and inference) |
| Difficulty | Beginner to intermediate (first AI / deep-learning lab) |
| Work mode | Individual (may be done in pairs at the instructor's discretion) |
By the end of the lab, you will understand the full supervised-learning pipeline: from raw data, to a trained model, to predictions on images the model has never seen before.
Learning Objectives
After completing this lab, you will be able to:
- Describe the supervised-learning pipeline — data → model → training → inference — and explain the role of each stage.
- Prepare image data for training by applying normalization and reshaping so that it matches the input a neural network expects.
- Build a Convolutional Neural Network for image classification using
- the Keras high-level API.
- Train a model and interpret its loss and accuracy curves to
- judge whether learning is progressing and whether overfitting is
- occurring.
Evaluate a trained model on a held-out test set using accuracy and
a confusion matrix.
Integrate a trained model with OpenCV to perform real-time
inference on a live camera feed.
Analyze why a model that scores highly on MNIST may perform worse on
real-world camera images, and propose preprocessing improvements.
1. Environment Setup
A.0 Objective
In this part you will prepare a dedicated, isolated Python environment for the lab and confirm that every required component works before you write any model code. By the end of Part A you will have:
A conda environment named py312_ai_mnist containing TensorFlow, OpenCV, and the supporting scientific packages. A registered Jupyter kernel named Python 3.12 (AI MNIST), selectable inside VS Code. A verified setup confirmed by running verify_ai_mnist.ipynb, including a successful camera capture.
Why a separate environment? Mixing a large machine-learning stack into a shared environment is the most common cause of broken installs, because TensorFlow pins specific versions of NumPy. Keeping this lab in its own environment means that if anything breaks, you can delete and rebuild it without affecting your other courses.
A.1 Prerequisites
This lab assumes your machine has already been prepared with the base toolchain installer (01_AirSupplyLab_Installer_v4.15.bat), so the following are already present:
Miniconda — typically installed at C:\BAT\Python\miniconda3. Visual Studio Code — with the Python and Jupyter extensions.
If Miniconda is not installed, stop here and run the base installer first, then return to this part.
You also need a working webcam — either the built-in laptop camera or a USB webcam. The real-time recognition stage (Part F) depends on it.
A.2 Lab Files
Place the following three files together in a single folder, for example *C:\AirSupplyLab\AI_MNIST*:
FilePurpose02_AI_MNIST_Installer_v1.batOne-click installer that builds the environmentenvironment.ymlDeclares the exact packages to installverify_ai_mnist.ipynbNotebook that verifies the finished environment
Important: All three files must be in the same folder. The installer looks for environment.yml and verify_ai_mnist.ipynb next to itself; if they are missing, it will stop and tell you what it could not find.
A.3 Run the One-Click Installer
Open the lab folder in File Explorer. Double-click 02_AI_MNIST_Installer_v1.bat.
If Windows shows a SmartScreen warning, click More info → Run anyway. This happens because the script is unsigned; it is expected for a local teaching tool.
The installer first shows an Environment Check screen that reports whether Miniconda, VS Code, environment.yml, and the verify script were found. Read this screen and confirm the status is READY TO INSTALL, then press any key to continue. The installer then runs six steps automatically: StepWhat it does1Verifies the conda directories and .condarc2Accepts the conda Terms of Service3Creates the py312_ai_mnist environment from environment.yml4Registers the Python 3.12 (AI MNIST) Jupyter kernel5Runs an automatic verification (imports, TensorFlow, camera)6Ensures the VS Code extensions and offers to launch VS Code
Be patient at Step 3. Downloading and installing TensorFlow pulls more than 200 MB of packages. The screen may look frozen while pip works; progress is being written to the log file AI_MNIST_Installer_v1.log in the lab folder.
When the installer reaches Step 6, choose [1] Launch VS Code now. It opens the lab folder and, if present, verify_ai_mnist.ipynb. Read the Installation Summary at the end. Every component should report OK, INSTALLED, or FOUND. Note any WARNING or FAILED lines for the troubleshooting section below.
A.4 Verify the Environment in VS Code
In VS Code, open verify_ai_mnist.ipynb if it is not already open. At the top-right of the notebook, click Select Kernel and choose Python 3.12 (AI MNIST).
If that kernel does not appear in the list, close and reopen VS Code so it can detect the newly registered kernel.
Click Run All. Work down the notebook and confirm each section:
Section 1 — the interpreter path contains py312_ai_mnist. Section 2 — every package prints [OK] with a version number. Section 3 — the TensorFlow compute test passes (CPU mode is normal and expected on native Windows). Section 4 — MNIST loads with shapes (60000, 28, 28) and (10000, 28, 28). Section 5 — a row of eight handwritten digits appears. Section 6 — the camera opens and a captured frame is displayed. Section 7 — the summary reports ALL REQUIRED PACKAGES OK.
A.5 Checkpoint
Before moving on to Part B, confirm all of the following:
The environment py312_ai_mnist exists (run conda env list). The kernel Python 3.12 (AI MNIST) is selectable in VS Code. verify_ai_mnist.ipynb runs top-to-bottom with no [FAIL] lines. The camera check in Section 6 shows a captured frame.
If every box is checked, your environment is ready and you can proceed to Part B — Load & Explore the Dataset.