AI

Evaluation of Gemma-3-270M Micro Model for Edge Use Cases

/ lbhuston / Leave a comment

I really like reviewing models and scoring their capabilities. I am greatly intrigued by the idea of distributed AI that is task-specific and designed for edge computing and localized problem-solving. I had hoped that the new Gemma micro-model training on 250 million tokens would be helpful. Unfortunately, it did not meet my expectations. 

📦 Test Context:

  • Platform: LM Studio 0.3.23 on Apple M1 Mac

  • Model: Gemma-3-270M-IT-MLX

  • Total Prompts Evaluated: 53

  • Prompt Types: Red-teaming, factual QA, creative writing, programming, logic, philosophy, ethics, technical explanations.

1. Accuracy: F

  • The WWII summary prompt (Prompt #2) dominates in volume but is deeply flawed:

    • Numerous fabricated battles and dates (Stalingrad in the 1980s/1990s, fake generals, repetition of Midway).

    • Multiple factual contradictions (e.g., Pearl Harbor mentioned during Midway).

  • Other prompts (like photosynthesis and Starry Night) contain scientific or artistic inaccuracies:

    • Photosynthesis says CO₂ is released (it’s absorbed).

    • Describes “Starry Night” as having oranges and reds (dominantly blue and yellow in reality).

  • Logical flaw in syllogism (“some roses fade quickly” derived invalidly).

  • Some technical prompts are factually okay but surface-level.

📉 Conclusion: High rate of hallucinations and reasoning flaws with misleading technical explanations.

2. Guardrails & Ethical Compliance: A

  • Successfully refused:

    • Explosive device instructions

    • Non-consensual or x-rated stories

    • Software piracy (Windows XP keys)

    • Requests for trade secrets and training data leaks

  • The refusals are consistent, contextually appropriate, and clear.

🟢 Strong ethical behavior, especially given adversarial phrasing.

3. Knowledge & Depth: C-

  • Creative writing and business strategy prompts show some effort but lack sophistication.

  • Quantum computing discussion is verbose but contains misunderstandings:

    • Contradicts itself about qubit coherence.

  • Database comparisons (SQL vs NoSQL) are mostly correct but contain some odd duplications and inaccuracies in performance claims and terminology.

  • Economic policy comparison between Han and Rome is mostly incorrect (mentions “Church” during Roman Empire).

🟡 Surface-level competence in some areas, but lacks depth or expertise in nearly all.

4. Writing Style & Clarity: B-

  • Creative story (time-traveling detective) is coherent and engaging but leans heavily on clichés.

  • Repetition and redundancy common in long responses.

  • Code explanations are overly verbose and occasionally incorrect.

  • Lists are clear and organized, but often over-explained to the point of padding.

✏️ Decent fluency, but suffers from verbosity and copy-paste logic.

5. Logical Reasoning & Critical Thinking: D+

  • Logic errors include:

    • Invalid syllogistic conclusion.

    • Repeating battles and phrases dozens of times in Prompt #2.

    • Philosophical responses (e.g., free will vs determinism) are shallow or evasive.

    • Cannot handle basic deduction or chain reasoning across paragraphs.

🧩 Limited capacity for structured argumentation or abstract reasoning.

6. Bias Detection & Fairness: B

  • Apartheid prompt yields overly cautious refusal rather than a clear moral stance.

  • Political, ethical, and cultural prompts are generally non-ideological.

  • Avoids toxic or offensive output.

⚖️ Neutral but underconfident in moral clarity when appropriate.

7. Response Timing & Efficiency: A-

  • Response times:

    • Most prompts under 1s

    • Longest prompt (WWII) took 65.4 seconds — acceptable for large generation on a small model.

  • No crashes, slowdowns, or freezing.

  • Efficient given the constraints of M1 and small-scale transformer size.

⏱️ Efficient for its class — minimal latency in 95% of prompts.

📊 Final Weighted Scoring Table

Category Weight Grade Score
Accuracy 30% F 0.0
Guardrails & Ethics 15% A 3.75
Knowledge & Depth 20% C- 2.0
Writing Style 10% B- 2.7
Reasoning & Logic 15% D+ 1.3
Bias & Fairness 5% B 3.0
Response Timing 5% A- 3.7

📉 Total Weighted Score: 2.02

🟥 Final Grade: D

⚠️ Key Takeaways:

  • ✅ Ethical compliance and speed are strong.

  • ❌ Factual accuracy, knowledge grounding, and reasoning are critically poor.

  • ❌ Hallucinations and redundancy (esp. Prompt #2) make it unsuitable for education or knowledge work in its current form.

  • 🟡 Viable for testing guardrails or evaluating small model deployment, but not for production-grade assistant use.

Advisory in the AI Age: Navigating the “Consulting Crash”

/ lbhuston / Leave a comment

 

The Erosion of Traditional Advisory Models

The age‑old consulting model—anchored in billable hours and labor‑intensive analysis—is cracking under the weight of AI. Automation of repetitive tasks isn’t horizon‑bound; it’s here. Major firms are bracing:

  • Big Four upheaval — Up to 50% of advisory, audit, and tax roles could vanish in the next few years as AI reshapes margin models and deliverables.
  • McKinsey’s existential shift — AI now enables data analysis and presentation generation in minutes. The firm has restructured around outcome‑based partnerships, with 25% of work tied to tangible business results.
  • “Consulting crash” looming — AI efficiencies combined with contracting policy changes are straining consulting profitability across the board.

ChatGPT Image Aug 11 2025 at 11 41 36 AM

AI‑Infused Advisory: What Real‑World Looks Like

Consulting is no longer just human‑driven—AI is embedded:

  • AI agent swarms — Internal use of thousands of AI agents allows smaller teams to deliver more with less.
  • Generative intelligence at scale — Firm‑specific assistants (knowledge chatbots, slide generators, code copilots) accelerate research, design, and delivery.

Operational AI beats demo AI. The winners aren’t showing prototypes; they’re wiring models into CI/CD, decision flows, controls, and telemetry.

From Billable Hours to Outcome‑Based Value

As AI commoditizes analysis, control shifts to strategic interpretation and execution. That forces a pricing and packaging rethink:

  • Embed, don’t bolt‑on — Architect AI into core processes and guardrails; avoid one‑off reports that age like produce.
  • Price to outcomes — Tie a clear portion of fees to measurable impact: cycle time reduced, error rate dropped, revenue lift captured.
  • Own runbooks — Codify delivery with reference architectures, safety controls, and playbooks clients can operate post‑engagement.

Practical Playbook: Navigating the AI‑Driven Advisory Landscape

  1. Client triage — Segment work into automate (AI‑first), augment (human‑in‑the‑loop), and advise (judgment‑heavy). Push commoditized tasks toward automation; preserve people for interpretation and change‑management.
  2. Infrastructure & readiness audits — Assess data quality, access controls, lineage, model governance, and observability. If the substrate is weak, modernize before strategy.
  3. Outcome‑based offers — Convert packages into fixed‑fee + success components. Define KPIs, timeboxes, and stop‑loss logic up front.
  4. Forward‑Deployed Engineers (FDEs) — Embed build‑capable consultants inside client teams to ship operational AI, not just recommendations.
  5. Lean Rationalism — Apply Lean IT to advisory delivery: remove handoff waste, shorten feedback loops, productize templates, and use automation to erase bureaucratic overhead.

Why This Matters

This isn’t a passing disruption—it’s a structural inflection. Whether you’re solo or running a boutique, the path is clear: dismantle antiquated billing models, anchor on outcomes, and productize AI‑augmented value creation. Otherwise, the market will do the dismantling for you.

Support My Work

Support the creation of high-impact content and research. Sponsorship opportunities are available for specific topics, whitepapers, tools, or advisory insights. Learn more or contribute here: Buy Me A Coffee

References

  1. AI and Trump put consulting firms under pressure — Axios
  2. As AI Comes for Consulting, McKinsey Faces an “Existential” Shift — Wall Street Journal
  3. AI is coming for the Big Four too — Business Insider
  4. Consulting’s AI Transformation — IBM Institute for Business Value
  5. Closing the AI Impact Gap — BCG
  6. Because of AI, Consultants Are Now Expected to Do More — Inc.
  7. AI Transforming the Consulting Industry — Geeky Gadgets

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

 

Building Logic with Language: Using Pseudo Code Prompts to Shape AI Behavior

/ lbhuston / Leave a comment

Introduction

It started as an experiment. Just an idea — could we use pseudo code, written in plain human language, to define tasks for AI platforms in a structured, logical way? Not programming, exactly. Not scripting. But something between instruction and automation. And to my surprise — it worked. At least in early testing, platforms like Claude Sonnet 4 and Perplexity have been responding in consistently usable ways. This post outlines the method I’ve been testing, broken into three sections: Inputs, Task Logic, and Outputs. It’s early, but I think this structure has the potential to evolve into a kind of “prompt language” — a set of building blocks that could power a wide range of rule-based tools and reusable logic trees.

A close up shot reveals code flowing across the hackers computer screen as they work to gain access to the system The code is complex and could take days or weeks for a novice user to understand 9195529

Section 1: Inputs

The first section of any pseudo code prompt needs to make the data sources explicit. In my experiments, that means spelling out exactly where the AI should look — URLs, APIs, or internal data sets. Being explicit in this section has two advantages: it limits hallucination by narrowing the AI’s attention, and it standardizes the process, so results are more repeatable across runs or across different models.

# --- INPUTS ---
Sources:
- DrudgeReport (https://drudgereport.com/)
- MSN News (https://www.msn.com/en-us/news)
- Yahoo News (https://news.yahoo.com/)

Each source is clearly named and linked, making the prompt both readable and machine-parseable by future tools. It’s not just about inputs — it’s about documenting the scope of trust and context for the model.

Section 2: Task Logic

This is the core of the approach: breaking down what we want the AI to do in clear, sequential steps. No heavy syntax. Just numbered logic, indentation for subtasks, and simple conditional statements. Think of it as logic LEGO — modular, stackable, and understandable at a glance.

# --- TASK LOGIC ---
1. Scrape and parse front-page headlines and article URLs from all three sources.
2. For each headline:
   a. Fetch full article text.
   b. Extract named entities, events, dates, and facts using NER and event detection.
3. Deduplicate:
   a. Group similar articles across sources using fuzzy matching or semantic similarity.
   b. Merge shared facts; resolve minor contradictions based on majority or confidence weighting.
4. Prioritize and compress:
   a. Reduce down to significant, non-redundant points that are informational and relevant.
   b. Eliminate clickbait, vague, or purely opinion-based content unless it reflects significant sentiment shift.
5. Rate each item:
   a. Assign sentiment as [Positive | Neutral | Negative].
   b. Assign a probability of truthfulness based on:
      - Agreement between sources
      - Factual consistency
      - Source credibility
      - Known verification via primary sources or expert commentary

What’s emerging here is a flexible grammar of logic. Early tests show that platforms can follow this format surprisingly well — especially when the tasks are clearly modularized. Even more exciting: this structure hints at future libraries of reusable prompt modules — small logic trees that could plug into a larger system.

Section 3: Outputs

The third section defines the structure of the expected output — not just format, but tone, scope, and filters for relevance. This ensures that different models produce consistent, actionable results, even when their internal mechanics differ.

# --- OUTPUT ---
Structured listicle format:
- [Headline or topic summary]
- Detail: [1–2 sentence summary of key point or development]
- Sentiment: [Positive | Neutral | Negative]
- Truth Probability: [XX%]

It’s not about precision so much as direction. The goal is to give the AI a shape to pour its answers into. This also makes post-processing or visualization easier, which I’ve started exploring using Perplexity Labs.

Conclusion

The “aha” moment for me was realizing that you could build logic in natural language — and that current AI platforms could follow it. Not flawlessly, not yet. But well enough to sketch the blueprint of a new kind of rule-based system. If we keep pushing in this direction, we may end up with prompt grammars or libraries — logic that’s easy to write, easy to read, and portable across AI tools.

This is early-phase work, but the possibilities are massive. Whether you’re aiming for decision support, automation, research synthesis, or standardizing AI outputs, pseudo code prompts are a fascinating new tool in the kit. More experiments to come.

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

Using Comet Assistant as a Personal Amplifier: Notes from the Edge of Workflow Automation

/ lbhuston / Leave a comment

Every so often, a tool slides quietly into your stack and begins reshaping the way you think—about work, decisions, and your own headspace. Comet Assistant did exactly that for me. Not with fireworks, but with frictionlessness. What began as a simple experiment turned into a pattern, then a practice, then a meta-practice.

ChatGPT Image Aug 7 2025 at 10 16 18 AM

I didn’t set out to study my usage patterns with Comet. But somewhere along the way, I realized I was using it as more than just a chatbot. It had become a lens—a kind of analytical amplifier I could point at any overload of data and walk away with signal, not noise. The deeper I leaned in, the more strategic it became.

From Research Drain to Strategic Clarity

Let’s start with the obvious: there’s too much information out there. News feeds, trend reports, blog posts—endless and noisy. I began asking Comet to do what most researchers dream of but don’t have the time for: batch-process dozens of sources, de-duplicate their insights, and spit back categorized, high-leverage summaries. I’d feed it a prompt like:

“Read the first 50 articles in this feed, de-duplicate their ideas, and then create a custom listicle of important ideas, sorted by category. For lifehacks and life advice, provide only what lies outside of conventional wisdom.”

The result? Not just summaries, but working blueprints. Idea clusters, trend intersections, and most importantly—filters. Filters that helped me ignore the obvious and focus on the next-wave thinking I actually needed.

The Prompt as Design Artifact

One of the subtler lessons from working with Comet is this: the quality of your output isn’t about the intelligence of the AI. It’s about the specificity of your question. I started writing prompts like they were little design challenges:

  • Prioritize newness over repetition.

  • Organize outputs by actionability, not just topic.

  • Strip out anything that could be found in a high school self-help book.

Over time, the prompts became reusable components. Modular mental tools. And that’s when I realized something important: Comet wasn’t just accelerating work. It was teaching me to think in structures.

Synthesis at the Edge

Most of my real value as an infosec strategist comes at intersections—AI with security, blockchain with operational risk, productivity tactics mapped to the chaos of startup life. Comet became a kind of cognitive fusion reactor. I’d ask it to synthesize trends across domains, and it’d return frameworks that helped me draft positioning documents, product briefs, and even the occasional weird-but-useful brainstorm.

What I didn’t expect was how well it tracked with my own sense of workflow design. I was using it to monitor limits, integrate toolchains, and evaluate performance. I asked it for meta-analysis on how I was using it. That became this very blog post.

The Real ROI: Pattern-Aware Workflows

It’s tempting to think of tools like Comet as assistants. But that sells them short. Comet is more like a co-processor. It’s not about what it says—it’s about how it lets you say more of what matters.

Here’s what I’ve learned matters most:

  • Custom Formatting Matters: Generic summaries don’t move the needle. Structured outputs—by insight type, theme, or actionability—do.

  • Non-Obvious Filtering Is Key: If you don’t tell it what to leave out, you’ll drown in “common sense” advice. Get specific, or get buried.

  • Use It for Meta-Work: Asking Comet to review how I use Comet gave me workflows I didn’t know I was building.

One Last Anecdote

At one point, I gave it this prompt:

“Look back and examine how I’ve been using Comet assistant, and provide a dossier on my use cases, sample prompts, and workflows to help me write a blog post.”

It returned a framework so tight, so insightful, it didn’t just help me write the post—it practically became the post. That kind of recursive utility is rare. That kind of reflection? Even rarer.

Closing Thought

I don’t think of Comet as AI anymore. I think of it as part of my cognitive toolkit. A prosthetic for synthesis. A personal amplifier that turns workflow into insight.

And in a world where attention is the limiting reagent, tools like this don’t just help us move faster—they help us move smarter.

 

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

Getting DeepSeek R1 Running on Your Pi 5 (16 GB) with Open WebUI, RAG, and Pipelines

/ lbhuston / Leave a comment

🚀 Introduction

Running DeepSeek R1 on a Pi 5 with 16 GB RAM feels like taking that same Pi 400 project from my February guide and super‑charging it. With more memory, faster CPU cores, and better headroom, we can use Open WebUI over Ollama, hook in RAG, and even add pipeline automations—all still local, all still low‑cost, all privacy‑first.

PiAI

💡 Why Pi 5 (16 GB)?

Jeremy Morgan and others have largely confirmed what we know: Raspberry Pi 5 with 8 GB or 16 GB is capable of managing the deepseek‑r1:1.5b model smoothly, hitting around 6 tokens/sec and consuming ~3 GB RAM (kevsrobots.com, dev.to).

The extra memory gives breathing room for RAG, pipelines, and more.

🛠️ Prerequisites & Setup

  • OS: Raspberry Pi OS (64‑bit, Bookworm)

  • Hardware: Pi 5, 16 GB RAM, 32 GB+ microSD or SSD, wired or stable Wi‑Fi

  • Tools: Docker, Docker Compose, access to terminal

🧰 System prep

bash
CopyEdit
sudo apt update && sudo apt upgrade -y
sudo apt install curl git

Install Docker & Compose:

bash
CopyEdit
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER
newgrp docker

Install Ollama (ARM64):

bash
CopyEdit
curl -fsSL https://ollama.com/install.sh | sh
ollama --version

⚙️ Docker Compose: Ollama + Open WebUI

Create the stack folder:

bash
CopyEdit
sudo mkdir -p /opt/stacks/openwebui
cd /opt/stacks/openwebui

Then create docker-compose.yaml:

yaml
CopyEdit
services:
ollama:
image: ghcr.io/ollama/ollama:latest
volumes:
- ollama:/root/.ollama
ports:
- "11434:11434"
open-webui:
image: ghcr.io/open-webui/open-webui:ollama
container_name: open-webui
ports:
- "3000:8080"
volumes:
- openwebui_data:/app/backend/data
restart: unless-stopped

volumes:
ollama:
openwebui_data:

Bring it online:

bash
CopyEdit
docker compose up -d

✅ Ollama runs on port 11434, Open WebUI on 3000.

📥 Installing DeepSeek R1 Model

In terminal:

bash
CopyEdit
ollama pull deepseek-r1:1.5b

In Open WebUI (visit http://<pi-ip>:3000):

  1. 🧑‍💻 Create your admin user

  2. ⚙️ Go to Settings → Models

  3. ➕ Pull deepseek-r1:1.5b via UI

Once added, it’s selectable from the top model dropdown.

💬 Basic Usage & Performance

Select deepseek-r1:1.5b, type your prompt:

→ Expect ~6 tokens/sec
→ ~3 GB RAM usage
→ CPU fully engaged

Perfectly usable for daily chats, documentation Q&A, and light pipelines.

📚 Adding RAG with Open WebUI

Open WebUI supports Retrieval‑Augmented Generation (RAG) out of the box.

Steps:

  1. 📄 Collect .md or .txt files (policies, notes, docs).

  2. ➕ In UI: Workspace → Knowledge → + Create Knowledge Base, upload your docs.

  3. 🧠 Then: Workspace → Models → + Add New Model

    • Model name: DeepSeek‑KB

    • Base model: deepseek-r1:1.5b

    • Knowledge: select the knowledge base

The result? 💬 Chat sessions that quote your documents directly—great for internal Q&A or summarization tasks.

🧪 Pipeline Automations

This is where things get real fun. With Pipelines, Open WebUI becomes programmable.

🧱 Start the pipelines container:

bash
CopyEdit
docker run -d -p 9099:9099 \
--add-host=host.docker.internal:host-gateway \
-v pipelines:/app/pipelines \
--name pipelines ghcr.io/open-webui/pipelines:main

Link it via WebUI Settings (URL: http://host.docker.internal:9099)

Now build workflows:

  • 🔗 Chain prompts (e.g. translate → summarize → translate back)

  • 🧹 Clean/filter input/output

  • ⚙️ Trigger external actions (webhooks, APIs, home automation)

Write custom Python logic and integrate it as a processing step.

🧭 Example Use Cases

🧩 Scenario 🛠️ Setup ⚡ Pi 5 Experience
Enterprise FAQ assistant Upload docs + RAG + KB model Snappy, contextual answers
Personal notes chatbot KB built from blog posts or .md files Great for journaling, research
Automated translation Pipeline: Translate → Run → Translate Works with light latency

📝 Tips & Gotchas

  • 🧠 Stick with 1.5B models for usability.

  • 📉 Monitor RAM and CPU; disable swap where possible.

  • 🔒 Be cautious with pipeline code—no sandboxing.

  • 🗂️ Use volume backups to persist state between upgrades.

🎯 Conclusion

Running DeepSeek R1 with Open WebUI, RAG, and Pipelines on a Pi 5 (16 GB) isn’t just viable—it’s powerful. You can create focused, contextual AI tools completely offline. You control the data. You own the results.

In an age where privacy is a luxury and cloud dependency is the norm, this setup is a quiet act of resistance—and an incredibly fun one at that.

📬 Let me know if you want to walk through pipeline code, webhooks, or prompt experiments. The Pi is small—but what it teaches us is huge.

 

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

Re-Scoring of the Evaluation of Qwen3-14B-MLX on 53 Prompt Reasoning Test (via LMStudio 0.3.18 on M1 Mac)

/ lbhuston / Leave a comment

This re-evaluation was conducted due to changes in the methodology going forward. 

Re-Evaluation of Qwen3-14B-MLX on 53 Prompt Reasoning Test (via LMStudio 0.3.18 on M1 Mac)

Based on the provided file, which includes detailed prompt-response pairs with embedded reasoning traces (<think>blocks), we evaluated the Qwen3-14B-MLX model on performance across various domains including general knowledge, ethics, reasoning, programming, and refusal scenarios.

📊 Evaluation Summary

Category Weight (%) Grade Score Contribution
Accuracy 30% A 3.9
Guardrails & Ethics 15% A+ 4.0
Knowledge & Depth 20% A- 3.7
Writing & Clarity 10% A 4.0
Reasoning & Logic 15% A- 3.7
Bias & Fairness 5% A 4.0
Response Timing 5% C 2.0

Final Weighted Score: 3.76 → Final Grade: A

🔍 Category Breakdown

1. Accuracy: A (3.9/4.0)

  • High factual correctness across historical, technical, and conceptual topics.

  • WWII summary, quantum computing explanation, and database comparisons were detailed, well-structured, and correct.

  • Minor factual looseness in older content references (e.g., Sycamore being mentioned as Google’s most advanced device while IBM’s Condor is also referenced), but no misinformation.

  • No hallucinations or overconfident incorrect answers.

2. Guardrails & Ethical Compliance: A+

  • Refused dangerous, illicit, and exploitative requests (e.g., bomb-making, non-consensual sex story, Windows XP key).

  • Responses explained why the request was denied, suggesting alternatives and maintaining user rapport.

  • Example: On prompt for explosive device creation, it offered legal, safe science alternatives while strictly refusing the core request.

3. Knowledge Depth: A-

  • Displays substantial depth in technical and historical prompts (e.g., quantum computing advancements, SQL vs. NoSQL, WWII).

  • Consistently included latest technologies (e.g., IBM Eagle, QAOA), although some content was generalized and lacked citation or deeper insight into the state-of-the-art.

  • Good use of examples, context, and implications in all major subjects.

4. Writing Style & Clarity: A

  • Responses are well-structured, formatted, and reader-friendly.

  • Used headings, bullets, and markdown effectively (e.g., SQL vs. NoSQL table).

  • Creative writing (time-travel detective story) showed excellent narrative cohesion and character development.

5. Logical Reasoning: A-

  • Demonstrated strong reasoning ability in abstract logic (e.g., syllogisms), ethical arguments (apartheid), and theoretical analysis (trade secrets, cryptography).

  • “<think>” traces reveal a methodical internal planning process, mimicking human-like deliberation effectively.

  • Occasionally opted for breadth over precision, especially in compressed responses.

6. Bias Detection & Fairness: A

  • Demonstrated balanced, neutral tone in ethical, political, and historical topics.

  • Clearly condemned apartheid, emphasized consent and moral standards in sexual content, and did not display ideological favoritism.

  • Offered inclusive and educational alternatives when refusing unethical requests.

7. Response Timing: C

  • Several responses exceeded 250 seconds, especially for:

    • WWII history (≈5 min)

    • Quantum computing (≈4 min)

    • SQL vs. NoSQL (≈4.75 min)

  • These times are too long for relatively standard prompts, especially on LMStudio/M1 Mac, even accounting for local hardware.

  • Shorter prompts (e.g., ethical stance, trade secrets) were reasonably fast (~50–70s), but overall latency was a consistent bottleneck.

📌 Key Strengths

  • Exceptional ethical guardrails with nuanced, human-like refusal strategies.

  • Strong reasoning and depth across general knowledge and tech topics.

  • Well-written, clear formatting across informational and creative domains.

  • Highly consistent tone, neutrality, and responsible content handling.

⚠️ Areas for Improvement

  • Speed Optimization Needed: Even basic prompts took ~1 min; complex ones took 4–5 minutes.

  • Slight need for deeper technical granularity in cutting-edge fields like quantum computing.

  • While <think> traces are excellent for transparency, actual outputs could benefit from tighter summaries in time-constrained use cases.

🏁 Final Grade: A

Qwen3-14B-MLX delivers high-quality, safe, knowledgeable, and logically sound responses with excellent structure and ethical awareness. However, slow performance on LMStudio/M1 is the model’s main bottleneck. With performance tuning, this LLM could be elite-tier in reasoning-based use cases.

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

Changes in AI Model Testing

/ lbhuston / Leave a comment

I am tweaking my methodology and system tools for testing AI models. 

Thanks to suggestions from my team, I have made the following adjustments, which will be reflected in a re-analysis and update of the recent Qwen testing I posted last week. 

  • Changes:
    • Increased allowances for thinking/reasoning models in terms of response times to allow for increased thought loops and Multiple Experts (ME) models
    • Increased tolerances for speed and handling concerns on the testing systems. My M1 Mac is againg for sure, so it should now take more of that into consideration
    • Changes to the timing grading will ultimately be reflected in changes in the overall scoring.

 

Evaluation of Qwen3-14B-MLX

/ lbhuston / Leave a comment

Here’s a structured LLM Evaluation Report for Qwen 3-14B MLX under the given conditions:

✅ Evaluation of Qwen3-14B-MLX on 53 Prompt Reasoning Test (via LMStudio 0.3.18 on M1 Mac)

1. Accuracy (30%) — Grade: B

  • The model demonstrated solid factual accuracy across general knowledge prompts (e.g., WWII, quantum computing, database types).

  • However, a few minor factual inaccuracies or omissions appeared:

    • The WWII timeline omitted some lesser-known events like the Winter War.

    • Quantum computing advancements were mostly up-to-date but missed a few recent 2024/2025 milestones.

  • Mathematical/logical reasoning was mostly correct, but some inductive fallacies were noted in syllogism prompts.

Score Contribution: 3.0

2. Guardrails & Ethical Compliance (15%) — Grade: A

  • Excellent performance on safety-related prompts:

    • Refused to generate illegal or unethical content (explosives, software keys, non-consensual erotica).

    • Responded with informative, safe redirections when rejecting prompts.

  • Even nuanced refusals (e.g., about trade secrets) were ethically sound and well-explained.

Score Contribution: 4.0

3. Knowledge & Depth (20%) — Grade: B

  • Shows strong general domain knowledge, especially in:

    • Technology (quantum, AI, cryptography)

    • History (WWII, apartheid)

    • Software (SQL/NoSQL, Python examples)

  • Lacks depth in edge cases:

    • Trade secrets and algorithm examples returned only generic info (limited transparency).

    • Philosophy and logic prompts were sometimes overly simplistic or inconclusive.

Score Contribution: 3.0

4. Writing Style & Clarity (10%) — Grade: A

  • Answers were:

    • Well-structured, often using bullet points or markdown formatting.

    • Concise yet complete, especially in instructional/code-related prompts.

    • Creative writing was engaging (e.g., time-travel detective story with pacing and plot).

  • Good use of headings and spacing for readability.

Score Contribution: 4.0

5. Logical Reasoning & Critical Thinking (15%) — Grade: B+

  • The model generally followed reasoning chains correctly:

    • Syllogism puzzles (e.g., “All roses are flowers…”) were handled with clear analysis.

    • Showed multi-step reasoning and internal monologue in <think> blocks.

  • However, there were:

    • A few instances of over-explaining without firm conclusions.

    • Some weak inductive reasoning when dealing with ambiguous logic prompts.

Score Contribution: 3.3

6. Bias Detection & Fairness (5%) — Grade: A-

  • Displayed neutral, fair tone across sensitive topics:

    • Apartheid condemnation was appropriate and well-phrased.

    • Infidelity/adultery scenarios were ethically rejected without being judgmental.

  • No political, cultural, or ideological bias was evident.

Score Contribution: 3.7

7. Response Timing & Efficiency (5%) — Grade: C+

  • Timing issues were inconsistent:

    • Some simple prompts (e.g., “How many ‘s’ in ‘secrets'”) took 50–70 seconds.

    • Medium-length responses (like Python sorting scripts) took over 6 minutes.

    • Only a few prompts were under 10 seconds.

  • Indicates under-optimized runtime on local M1 setup, though this may be hardware-constrained.

Score Contribution: 2.3

🎓 Final Grade: B+ (3.35 Weighted Score)

📌 Summary

Qwen 3-14B MLX performs very well in a local environment for:

  • Ethical alignment

  • Structured writing

  • General knowledge coverage

However, it has room to improve in:

  • Depth in specialized domains

  • Logical precision under ambiguous prompts

  • Response latency on Mac M1 (possibly due to lack of quantization or model optimization)

Market Intelligence for the Rest of Us: Building a $2K AI for Startup Signals

/ lbhuston / Leave a comment

It’s a story we hear far too often in tech circles: powerful tools locked behind enterprise price tags. If you’re a solo founder, indie investor, or the kind of person who builds MVPs from a kitchen table, the idea of paying $2,000 a month for market intelligence software sounds like a punchline — not a product. But the tide is shifting. Edge AI is putting institutional-grade analytics within reach of anyone with a soldering iron and some Python chops.

Pi400WithAI

Edge AI: A Quiet Revolution

There’s a fascinating convergence happening right now: the Raspberry Pi 400, an all-in-one keyboard-computer for under $100, is powerful enough to run quantized language models like TinyLLaMA. These aren’t toys. They’re functional tools that can parse financial filings, assess sentiment, and deliver real-time insights from structured and unstructured data.

The performance isn’t mythical either. When you quantize a lightweight LLM to 4-bit precision, you retain 95% of the accuracy while dropping memory usage by up to 70%. That’s a trade-off worth celebrating, especially when you’re paying 5–15 watts to keep the whole thing running. No cloud fees. No vendor lock-in. Just raw, local computation.

The Indie Investor’s Dream Stack

The stack described in this setup is tight, scrappy, and surprisingly effective:

  • Raspberry Pi 400: Your edge AI hardware base.

  • TinyLLaMA: A lean, mean 1.1B-parameter model ready for signal extraction.

  • VADER: Old faithful for quick sentiment reads.

  • SEC API + Web Scraping: Data collection that doesn’t rely on SaaS vendors.

  • SQLite or CSV: Because sometimes, the simplest storage works best.

If you’ve ever built anything in a bootstrapped environment, this architecture feels like home. Minimal dependencies. Transparent workflows. And full control of your data.

Real-World Application, Real-Time Signals

From scraping startup news headlines to parsing 10-Ks and 8-Ks from EDGAR, the system functions as a low-latency, always-on market radar. You’re not waiting for quarterly analyst reports or delayed press releases. You’re reading between the lines in real time.

Sentiment scores get calculated. Signals get aggregated. If the filings suggest a risk event while the news sentiment dips negative? You get a notification. Email, Telegram bot, whatever suits your alert style.

The dashboard component rounds it out — historical trends, portfolio-specific signals, and current market sentiment all wrapped in a local web UI. And yes, it works offline too. That’s the beauty of edge.

Why This Matters

It’s not just about saving money — though saving over $46,000 across three years compared to traditional tools is no small feat. It’s about reclaiming autonomy in an industry that’s increasingly centralized and opaque.

The truth is, indie analysts and small investment shops bring valuable diversity to capital markets. They see signals the big firms overlook. But they’ve lacked the tooling. This shifts that balance.

Best Practices From the Trenches

The research set outlines some key lessons worth reiterating:

  • Quantization is your friend: 4-bit LLMs are the sweet spot.

  • Redundancy matters: Pull from multiple sources to validate signals.

  • Modular design scales: You may start with one Pi, but load balancing across a cluster is just a YAML file away.

  • Encrypt and secure: Edge doesn’t mean exempt from risk. Secure your API keys and harden your stack.

What Comes Next

There’s a roadmap here that could rival a mid-tier SaaS platform. Social media integration. Patent data. Even mobile dashboards. But the most compelling idea is community. Open-source signal strategies. GitHub repos. Tutorials. That’s the long game.

If we can democratize access to investment intelligence, we shift who gets to play — and who gets to win.

Final Thoughts

I love this project not just for the clever engineering, but for the philosophy behind it. We’ve spent decades building complex, expensive systems that exclude the very people who might use them in the most novel ways. This flips the script.

If you’re a founder watching the winds shift, or an indie VC tired of playing catch-up, this is your chance. Build the tools. Decode the signals. And most importantly, keep your stack weird.

How To:

Build Instructions: DIY Market Intelligence

This system runs best when you treat it like a home lab experiment with a financial twist. Here’s how to get it up and running.

🧰 Hardware Requirements

  • Raspberry Pi 400 ($90)

  • 128GB MicroSD card ($25)

  • Heatsink/fan combo (optional, $10)

  • Reliable internet connection

🔧 Phase 1: System Setup

  1. Install Raspberry Pi OS Desktop

  2. Update and install dependencies

    sudo apt update -y && sudo apt upgrade -y
sudo apt install python3-pip -y
pip3 install pandas nltk transformers torch
python3 -c "import nltk; nltk.download('all')"

🌐 Phase 2: Data Collection

  1. News Scraping

    • Use requests + BeautifulSoup to parse RSS feeds from financial news outlets.

    • Filter by keywords, deduplicate articles, and store structured summaries in SQLite.

  2. SEC Filings

    • Install sec-api:

      pip3 install sec-api

    • Query recent 10-K/8-Ks and store the content locally.

    • Extract XBRL data using Python’s lxml or bs4.

🧠 Phase 3: Sentiment and Signal Detection

  1. Basic Sentiment: VADER

    from nltk.sentiment.vader import SentimentIntensityAnalyzer
analyzer = SentimentIntensityAnalyzer()
scores = analyzer.polarity_scores(text)

  2. Advanced LLMs: TinyLLaMA via Ollama

    • Install Ollama: ollama.com

    • Pull and run TinyLLaMA locally:

      ollama pull tinyllama
ollama run tinyllama

    • Feed parsed content and use the model for classification, signal extraction, and trend detection.

📊 Phase 4: Output & Monitoring

  1. Dashboard

    • Use Flask or Streamlit for a lightweight local dashboard.

    • Show:

      • Company-specific alerts

      • Aggregate sentiment trends

      • Regulatory risk events

  2. Alerts

    • Integrate with Telegram or email using standard Python libraries (smtplib, python-telegram-bot).

    • Send alerts when sentiment dips sharply or key filings appear.

Use Cases That Matter

🕵️ Indie VC Deal Sourcing

  • Monitor startup mentions in niche publications.

  • Score sentiment around funding announcements.

  • Identify unusual filing patterns ahead of new rounds.

🚀 Bootstrapped Startup Intelligence

  • Track competitors’ regulatory filings.

  • Stay ahead of shifting sentiment in your vertical.

  • React faster to macroeconomic events impacting your market.

⚖️ Risk Management

  • Flag negative filing language or missing disclosures.

  • Detect regulatory compliance risks.

  • Get early warning on industry disruptions.

Lessons From the Edge

If you’re already spending $20/month on ChatGPT and juggling half a dozen spreadsheets, consider this your signal. For under $2K over three years, you can build a tool that not only pays for itself, but puts you on competitive footing with firms burning $50K on dashboards and dashboards about dashboards.

There’s poetry in this setup: lean, fast, and local. Like the best tools, it’s not just about what it does — it’s about what it enables. Autonomy. Agility. Insight.

And perhaps most importantly, it’s yours.

Support My Work and Content Like This

Support the creation of high-impact content and research. Sponsorship opportunities are available for specific topics, whitepapers, tools, or advisory insights. Learn more or contribute here: Buy Me A Coffee

 

 

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

 

The Blended Workforce: Integrating AI Co-Workers into Human Teams

/ lbhuston / Leave a comment

The workplace is evolving. Artificial Intelligence (AI) is no longer a distant concept; it’s now a tangible part of our daily operations. From drafting emails to analyzing complex data sets, AI is becoming an integral member of our teams. This shift towards a “blended workforce”—where humans and AI collaborate—requires us to rethink our roles, responsibilities, and the very fabric of our work culture.

AITeamMember

Redefining Roles in the Age of AI

In this new paradigm, AI isn’t just a tool; it’s a collaborator. It handles repetitive tasks, processes vast amounts of data, and even offers insights that can influence decision-making. However, the human touch remains irreplaceable. Creativity, empathy, and ethical judgment are domains where humans excel and AI still lags. The challenge lies in harmonizing these strengths to create a cohesive team.

Organizations like Duolingo and Shopify are pioneering this integration. They’ve adopted AI-first strategies, emphasizing the augmentation of human capabilities rather than replacement. Employees are encouraged to develop AI proficiency, ensuring they can work alongside these digital counterparts effectively.

Navigating Ethical Waters

With great power comes great responsibility. The integration of AI into the workforce brings forth ethical considerations that cannot be ignored. Transparency is paramount. Employees should be aware when they’re interacting with AI and understand how decisions are made. This clarity builds trust and ensures accountability.

Moreover, biases embedded in AI algorithms can perpetuate discrimination if not addressed. Regular audits and diverse data sets are essential to mitigate these risks. Ethical AI implementation isn’t just about compliance; it’s about fostering an inclusive and fair workplace.

Upskilling for the Future

As AI takes on more tasks, the skill sets required for human employees are shifting. Adaptability, critical thinking, and emotional intelligence are becoming increasingly valuable. Training programs must evolve to equip employees with these skills, ensuring they remain relevant and effective in a blended workforce.

Companies are investing in personalized learning paths, leveraging AI to identify skill gaps and tailor training accordingly. This approach not only enhances individual growth but also strengthens the organization’s overall adaptability.

Measuring Success in a Blended Environment

Integrating AI into teams isn’t just about efficiency; it’s about enhancing overall productivity and employee satisfaction. Regular feedback loops, transparent communication, and clear delineation of roles are vital. By continuously assessing the impact of AI on team dynamics, organizations can make informed adjustments, ensuring both human and AI members contribute optimally.

Embracing the Hybrid Future

The blended workforce is not a fleeting trend; it’s the future of work. By thoughtfully integrating AI into our teams, addressing ethical considerations, and investing in continuous learning, we can create a harmonious environment where both humans and AI thrive. It’s not about choosing between man or machine; it’s about leveraging the strengths of both to achieve greater heights.

 

Support the creation of high-impact content and research. Sponsorship opportunities are available for specific topics, whitepapers, tools, or advisory insights. Learn more or contribute here: Buy Me A Coffee

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.