Introduction

I tend to explore new technologies hands-on before diving deeper into their intricacies, and AI is no exception. While AI has seamlessly integrated into my daily routine, I initially struggled to grasp some basic concepts. This passivity felt inadequate; I recognized the need to proactively educate myself—after all, AI is here to stay.

This article outlines my personal integration of AI into my daily activities and highlights essential concepts that can help us stay updated with advancements. The ideas presented here are primarily drawn from two sources:

1. Ollama library
2. Models — Top Weekly | OpenRouter

If you find these foundational concepts tedious, feel free to skip them (though I wouldn’t recommend it).

Integrating AI into Daily Life

Before we proceed, allow me to share my computer setup:

• MacOS + M1 Pro 32G

I've organized this section for you, allowing you to choose a scenario that piques your interest.

#### Writing: Obsidian

Utilize GitHub — logancyang/obsidian-copilot: A ChatGPT Copilot in Obsidian.

Features and scenarios include:

1. Translate text into English without visiting an AI website.
2. Content expansion.
3. Support for localized LLMs like Ollama.

Example: Using Emojify selection

For more information:

1. video: Introducing Copilot for Obsidian — YouTube
2. doc: Copilot for Obsidian | Obsidian Copilot

#### Coding: VSCode

Use the plugin: GitHub — continuedev/continue.

Features and scenarios include:

1. Code explanations.
2. Code completion.
3. Compatibility with localized LLMs like Ollama.
4. Usability with Jetbrain IDEs.

For more information:

1. doc: Introduction | Continue

#### Browsing: Chrome

##### ChatGPTBox

Usage scenarios include:

1. Providing answers for Chrome searches.
2. Summarizing web pages and videos.

Example with Gemini:

1. Search query
2. Video summary

#### Page Assist — A Web UI for Local AI Models

Usage scenarios include:

1. Support for localized LLMs like Ollama.
2. Assistance with article-related inquiries.

Mail and Notes

Reference: System-wide text summarization using Ollama and AppleScript for configuration.

Note: Upon first use, run the following command to install the model (further details on installing Ollama will follow).

ollama run mistral

Usage scenarios and features include:

1. Your notes and emails.
2. Use of localized LLMs like Ollama.

#### Mobile (iOS) Browsing

I occasionally use the Arc browser on my phone for searches, primarily leveraging two features:

1. Summarize entire pages with a pinch: Get the gist, in a pinch. Pinch to Summarize, now available in Arc Search.
2. Browses for you: Arc Search for iPhone | The fastest way to search — YouTube.

Model Selection

#### Non-Local Models

ChatGPT, Claude.ai, Bing, Gemini, Poe, OpenRouter, and others are available for experimentation. Most online content regarding AI primarily focuses on these models, so I won’t go into detail.

#### Local Models

##### Ollama

GitHub — ollama/ollama: Get started with Llama 3, Mistral, Gemma, and other large language models. Ollama is an open-source service tool designed for managing, reasoning with models, and offering APIs for other applications. It boasts a vibrant ecosystem.

The model files for Ollama can be found at: ?/.ollama/models, accessible via the command:

ollama show starcoder2:3b --modelfile

Additional installation and usage commands include:

ollama -h ollama run -h

To download the initial model, likely Llama3, refer to the article: Introducing Meta Llama 3: The most capable openly available LLM to date.

##### LM Studio

LM Studio — Discover, download, and run local LLMs. While Ollama is excellent, it can be complex for regular users. LM Studio offers a user-friendly UI for seamless engagement, although it is not open-source and currently free to use.

LM Studio enables model searching, downloading, managing, and running from one interface, accommodating chat, multiple model sessions, and Local LLM Server. The model directory is located at: ~/.cache/lm-studio/models/.

I’ve tested it and found it quite effective, although my computer struggles to run it frequently.

Basic AI Concepts

First, thank you for sticking around this long. As a programmer, my experience with AI tools has greatly enhanced my productivity, and I hope the same for you.

If you're using these AI tools for the first time, you may wonder: What model should I install? This requires understanding related concepts and making informed decisions based on your specific needs.

#### LLM & Prompting

Large Language Model — A widely known term; therefore, I won’t elaborate. The primary models we can run locally include Llama from Meta, specifically Llama2 and Llama3. Competing models also exist, such as those from Mistral AI.

Prompt engineering — A promising technique that could change how we interact with LLMs.

For a deeper dive, I recommend reviewing how-to guides on prompting and experimenting as per your requirements.

#### Parameters

Origin of the term: available in both 8B and 70B parameter sizes

“B” signifies billions of parameters, generally seen as a quality benchmark, with more parameters typically leading to superior performance. For instance, Llama3 offers models in 8B and 70B sizes:

#### Quantization

Origin of the term: quantization Q4_0

If a full-size model occupies 2 bytes per parameter (usually 16-bit floating-point numbers or FP16), a 3B model's actual size would be 6GB. If we were to quantize this model from FP16 (2 bytes) to 8-bit (1 byte), the size would reduce to 3GB.

The 8-bit refers to Q8, and the original text’s Q4 likely pertains to quantization at 4-bit. With Q4 technology, the size would drop to 1.5GB.

Refer to the quantization how-to guides for further insights.

#### Fine-Tuning

Origin of the term: Llama 3 instruction-tuned models are optimized for dialogue/chat applications

Fine-tuning involves additional training on top of pre-trained models to tailor them to specific tasks or datasets, typically requiring less computational power and time since the model already possesses extensive general knowledge.

Refer to the fine-tuning how-to guides for further details.

#### Context

Origin of the term: 200,000 context

This refers to the input information range a model considers when generating outputs or predictions. Since LLMs do not retain historical dialogue, all relevant past dialogue must be included to ensure accurate responses. Essentially, “context” describes the text segment or information range visible to a model while processing a task.

For models like GPT-3, the context window could be 2048 tokens or more, meaning the model only processes tokens within that window when producing outputs.

#### Additional Concepts: Temperature, Top_p, Top_k…

Reference: Mistral: Mistral 7B Instruct — Recommended Parameters | OpenRouter

Conclusion

Before we part ways, let me ask: Do you have a clearer understanding of the concepts discussed? If not, don't worry—you can leverage AI to enhance your learning. Trust me, once you start engaging with it, your comprehension will deepen.

This article primarily serves two purposes:

1. I’ve highlighted some readily available AI tools.
2. I’ve covered basic AI concepts.

I hope this assists you in selecting the right model and enables AI to work more effectively for you. If this also sparks your interest in AI and helps you stay abreast of emerging trends, that would be even better.

By the way, many scenarios mentioned can be utilized with local models like Ollama, which aids in safeguarding our privacy—how fantastic is that?

I believe this trend will continue into the future, leading to customized AI models tailored to you. Let's look forward to that together!

Additional Reading

1. What are the strengths of various AI models from different manufacturers?
2. Stay updated with AI news and learn about the current state of AI development.
3. Utilize APIs provided by AI manufacturers to develop code.

Enjoy the journey!

References

System-wide text summarization using Ollama and AppleScript.

More Series Articles about Stay Ahead of The Curve?

I’m Wesley, eager to share insights from the programming world.

Don’t forget to follow me for more informative content, or share this with those who might find it useful—it would greatly support me.

Catch you in the next article!