CSTTP Virtual Lab, Estevan and Ocean Man together. Build a tiny language model, see how the big ones are trained, and talk through what this means for our projects.
ChatGPT, Copilot and Claude all do one thing over and over: look at the text so far and guess what comes next. Then they add that guess and guess again. That is the whole trick, done with billions of examples.
Try it as a class. What comes next?
You guessed from everything you have read and heard before. A model does the same thing, but its "before" is its training data.
This model learns by counting which word follows which in the training text. Real LLMs are far more complex, but the core idea is the same: patterns in the data become predictions.
Real LLMs split text into word pieces, not whole words. A word they rarely saw gets chopped into many small pieces, which is one reason they struggle with languages that are rare online.
Click a prediction to keep going.
Press it a few times. Same model, different rolls of the dice.
| Stage | What happens | What it means for us |
|---|---|---|
| 1. Pretraining | The model reads a huge pile of text, mostly from the internet and books, and learns to predict the next word piece. This takes months and costs millions of dollars. | The model only knows what was in that pile. Languages with little text online, like Nakoda, barely show up. |
| 2. Fine tuning | A trained model gets extra training on a smaller, focused set of examples, like questions and good answers. | This is how a model could be taught a specific skill or style using our own data. It changes the model itself. |
| 3. Feedback | People rate answers, and the model is nudged toward the ones people prefer. | Who gives the feedback shapes how the model behaves. |
| 4. Use | We type a prompt and the model predicts its answer word by word. | It can sound sure and still be wrong. Someone who knows the topic needs to check. |
The Ocean Man project idea is to scan Nakoda textbooks (with permission), use AI to help build a word database, and have Elders and Knowledge Keepers check the results. There are two main ways to bring the textbooks to a model.
Train the model further on the textbook text so the patterns become part of the model.
Keep the textbook text in a database. The model searches it and quotes from it when answering.
Either way, the model runs on college computers, not a cloud service, unless Ocean Man agrees otherwise. And either way, Elders and Knowledge Keepers decide what is right.
AI helps turn scanned textbooks into a searchable word database and learning tools. Elders and Knowledge Keepers check every entry. The community controls the data.
AI checks event posts before they go live: a missing date, a time that makes no sense, no link to a source. A person still approves each event.
Dispatch data is confidential, so AI models run on our own computers instead of a cloud service. Dananji's research looks at what the data can predict to help dispatch work.
AI could help sort interview notes into common themes about why health staff stay or leave. Notes have names removed, stay on secure college systems, and AI never scores or judges a person.