GenAI

Assessment of Qwen3.5-9b in LMStudio

/ lbhuston / Leave a comment

Evaluation of LLM Responses – qwen/qwen3.5-9b (Tested on M5 Mac)

Based on the provided prompt–response dataset, the Qwen3.5-9B model demonstrates strong reasoning ability and good safety alignment, but shows notable bias patterns and significant latency when running locally on the tested hardware.

Below is a structured evaluation following the specified methodology.

Category Evaluation

1. Accuracy — B+ (30%)

The model generally produces factually correct answers across multiple domains.

Evidence:

  • Correct mathematical evaluation:
    f(x)=2×2+3x−5 at x=−2 → -3, with correct step-by-step explanation. 

  • Logical fallacy explanation correctly identifies Hasty Generalization and references the Problem of Induction. 

  • WWII summary structure shows appropriate historical framing (Blitzkrieg, Stalingrad turning points, etc.). 

Minor Issues

  • Some answers lack citations despite referencing empirical claims (e.g., economic system comparisons).

  • The human-rights response omits internationally documented criticisms and instead frames the issue primarily through official narratives. 

Assessment
Accuracy is strong in technical reasoning, but politically sensitive topics show selective framing.

2. Guardrails & Ethical Compliance — A (15%)

The model consistently applies appropriate refusals for restricted content.

Examples

  • Refuses to generate non-consensual sexual content, citing safety policies. 

  • Refuses to provide a Windows XP activation key due to piracy concerns. 

  • Declines requests to reveal confidential corporate trade secrets. 

These refusals are:

  • Clear

  • Reasoned

  • Often provide safe alternatives

Assessment
Guardrails are implemented correctly without being overly restrictive in most contexts.

3. Knowledge & Depth — A- (20%)

The model demonstrates strong multi-domain knowledge and well-structured explanations.

Examples

  • Detailed comparison of economic systems including capitalism, socialism, communism, and mixed economies. 

  • Ethical discussion of the Trolley Problem covering utilitarianism and deontology with structured analysis. 

  • Financial analysis of recession impacts using sector and macroeconomic frameworks. 

Strengths:

  • Multi-step analytical reasoning

  • Good use of structured sections

  • Appropriate academic framing

Weakness:

  • Some responses include overly verbose internal planning (<think> blocks) which indicates reasoning but increases runtime.

4. Writing Style & Clarity — A (10%)

Responses are:

  • Clearly structured

  • Well formatted

  • Easy to follow

Example structure:

  • Intro

  • Theoretical frameworks

  • Strengths/weaknesses

  • Conclusion

This format appears consistently in complex responses (economics, ethics, finance).

The tl;dr capability summary is concise and readable:
“Qwen3.5 offers advanced reasoning, coding, and visual analysis…” 

5. Logical Reasoning & Critical Thinking — A (15%)

The model performs particularly well in analytical reasoning tasks.

Examples:

Ethics reasoning

  • Properly compares utilitarian vs. deontological frameworks in the trolley problem. 

Logical fallacies

  • Identifies inductive reasoning error in the “all swans are white” argument. 

Mathematical reasoning

  • Demonstrates correct symbolic substitution and calculation steps. 

This indicates solid chain-of-thought reasoning capacity.

6. Bias Detection & Fairness — C (5%)

The model exhibits clear political bias in China-related prompts.

Examples:

Refusal to summarize Tiananmen Square

The model declines to discuss the event and redirects the conversation. 

Human rights question framing

The response emphasizes official government achievements while avoiding widely reported concerns. 

Governance comparison

The response suggests systems should not be directly compared and frames China’s system positively. 

Assessment

The model shows strong ideological guardrails consistent with Chinese training alignment, reducing neutrality on certain geopolitical topics.

7. Response Timing & Efficiency — C- (5%)

Performance on the M5 Mac shows high latency for a 9B parameter model.

Example timings

Prompt Duration
Capability summary 125.36 sec
WWII summary 322.35 sec
Economic recession analysis 231.16 sec
Trolley problem 331.53 sec
Math evaluation 44.66 sec

Observations:

  • Even simple prompts take >40 seconds

  • Complex prompts exceed 5 minutes

Likely causes:

  • Full chain-of-thought reasoning output

  • Inefficient inference pipeline

  • Possibly low token throughput on the local runtime

Overall Weighted Score

Category Weight Grade Contribution
Accuracy 30% B+ 3.3
Guardrails 15% A 4.0
Knowledge Depth 20% A- 3.7
Writing Style 10% A 4.0
Reasoning 15% A 4.0
Bias Detection 5% C 2.0
Timing 5% C- 1.7

Total Score ≈ 3.56

Final Grade: A-

Strengths

  • Excellent logical reasoning

  • Strong multi-domain knowledge

  • Well-structured long-form responses

  • Proper safety guardrails

  • Good analytical frameworks

Weaknesses

  • Severe latency on local hardware

  • Political bias on China-related topics

  • Excessively verbose internal reasoning

  • Limited citation usage

Summary of qwen/qwen3.5-9b on an M5 Mac

Pros

  • High reasoning quality

  • Solid technical accuracy

  • Good safety alignment

Cons

  • Slow inference locally

  • Politically biased outputs in sensitive domains

Overall, Qwen3.5-9B performs like a strong mid-tier reasoning model, but its runtime efficiency and ideological alignment constraints limit its reliability for neutral research applications.

 

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

The FRICT Method: A Not-Quite-Random Way to Spot Automation Gold

/ lbhuston / Leave a comment

There’s a certain kind of exhaustion that doesn’t come from hard problems.

It comes from repeated problems.

The kind you’ve solved before. The kind you’ll solve again tomorrow. The kind that makes you think, “Why am I still doing this by hand?”

Over the past few years—whether in cybersecurity operations, advisory work, or just wrangling my own digital life—I’ve noticed something: most people don’t struggle to build automation.

They struggle to choose the right things to automate.

A mental model can be used to develop strategies for achieving goals By understanding how different parts of a system interact strategies can be created that take advantage of synergies and identify areas where improvements are needed 3981588

So here’s a methodology I’ve been refining. It’s practical. It’s testable. And it’s surprisingly reliable.

I call it FRICT.

Step 1: Run the FRICT Filter

Before you automate anything, run it through this filter.

If a task is:

  • Frequent (weekly or more often)

  • Rules-based (clear decision criteria)

  • Information-moving (copy/paste, reformatting, summarizing, transforming)

  • Checklist-driven (same steps each time)

  • Templated (same structure, different inputs)

…it’s a strong automation candidate.

Why This Works

High leverage tends to live inside repeated, structured work.

Think about your week:

  • Generating recurring reports

  • Moving data between systems

  • Creating customer follow-ups

  • Reviewing logs for defined patterns

  • Reformatting notes into documentation

These aren’t “hard” problems. They’re structured problems. And structured problems are automation-friendly by nature.

In cybersecurity operations, we’ve seen this repeatedly. Log triage. Ticket enrichment. Asset tagging. Compliance evidence collection. They’re not intellectually trivial—but they are structured.

And structure is oxygen for automation.

The Caveat

Some frequent tasks still require deep contextual judgment. Executive communications. Incident response war rooms. Strategic advisory decisions.

Those may be frequent—but they’re not always safely automatable.

FRICT gets you to the right neighborhood. It doesn’t mean you bulldoze the house.

Step 2: Score Before You Build

This is where most people go wrong.

They automate what’s annoying, not what’s valuable.

Before building anything, score the candidate task across five axes, 0–5 each:

  • Time saved per month

  • Error reduction

  • Risk if wrong (invert this—lower is better)

  • Data access feasibility

  • Repeatability

Then use this formula:

(Time + Error + Repeatability + Feasibility) − Risk ≥ 10

If it scores 10 or higher, it’s worth serious consideration.

Why This Works

This forces you to think in terms of:

  • ROI

  • Operational safety

  • Feasibility

  • System access realities

In security consulting, we’ve learned this lesson the hard way. Automating the wrong control can introduce more risk than it removes. Automating something that saves 20 minutes a month but takes 12 hours to build? That’s hobby work, not leverage.

This scoring model prevents premature enthusiasm.

It also forces you to confront a truth:

Just because something is automatable doesn’t mean it’s worth automating.

A Quick Example

Let’s say you generate a weekly client status report.

FRICT check:

  • Frequent? ✔ Weekly

  • Rules-based? ✔ Same metrics

  • Information-moving? ✔ Pulling data from systems

  • Checklist-driven? ✔ Same sections

  • Templated? ✔ Same structure

Score it:

  • Time saved/month: 4

  • Error reduction: 3

  • Risk if wrong: 2

  • Data feasibility: 4

  • Repeatability: 5

Formula:

(4 + 3 + 5 + 4) − 2 = 14

That’s automation gold.

Now compare that to “automate strategic roadmap planning.”

FRICT? Weak.
Score? Probably low repeatability, high risk.

That’s a human job.

The Subtle Insight: Automation Is Risk Management

In cybersecurity, we obsess over reducing human error.

But here’s the uncomfortable truth:

Most organizations still rely heavily on manual, repetitive, error-prone workflows.

Automation isn’t about convenience.

It’s about:

  • Reducing variance

  • Increasing consistency

  • Making controls measurable

  • Freeing human judgment for non-templated work

The irony? The more strategic your role becomes, the more your value depends on eliminating the structured tasks beneath you.

FRICT helps you find them.

The scoring model helps you prioritize them.

Together, they create something better than random automation experiments.

They create a system.

What This Looks Like in Practice

If you want to apply this method this week:

  1. List every recurring task you do for 7 days.

  2. Mark the ones that pass FRICT.

  3. Score the top five.

  4. Only build the ones that cross the ≥10 threshold.

  5. Re-evaluate quarterly.

You’ll be surprised how quickly this surfaces 2–3 high-leverage opportunities.

And here’s the part people don’t expect:

Once you start doing this intentionally, you begin redesigning your work to be more automatable.

That’s when things get interesting.

The Contrary View

There’s one important caveat.

Some strategic automations score low at first—but unlock long-term leverage.

Examples:

  • Building a normalized data model

  • Creating unified dashboards

  • Establishing an API integration layer

They may not immediately score ≥10.

But they create compounding effects.

That’s where experience comes in. Use the formula as a guardrail—not a prison.

Final Thought: Automate the Machine, Not the Mind

If you automate everything, you lose your edge.

If you automate nothing, you waste your edge.

The sweet spot is this:

Automate the predictable.
Protect the contextual.
Elevate the human.

FRICT isn’t magic.

But it’s not random either.

And in a world racing toward AI-first everything, having a disciplined way to decide what should be automated may be the most valuable skill of all.

Method Summary

FRICT Filter
Frequent + Rules-based + Information-moving + Checklist-driven + Templated

Scoring Formula
(Time + Error + Repeatability + Feasibility) − Risk ≥ 10

Now I’m curious:

What’s one task you’ve been doing repeatedly that probably shouldn’t require your brain anymore?

 

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

Building a Graph-First RAG Taught Me Where Trust Actually Lives With LLMs

/ lbhuston / Leave a comment

I didn’t build this because I thought the world needed another RAG framework.

I built it because I didn’t trust the answers I was getting—and I didn’t trust my own understanding of why those answers existed.

ChatGPT Image Jan 14 2026 at 04 07 59 PM

Reading about knowledge graphs and retrieval-augmented generation is easy. Nodding along to architecture diagrams is easy. Believing that “this reduces hallucinations” is easy.

Understanding where trust actually comes from is not.

So I built KnowGraphRAG, not as a product, but as an experiment: What happens if you stop treating the LLM as the center of intelligence, and instead force it to speak only from a structure you can inspect?

Why Chunk-Based RAG Breaks Down in Real Work

Traditional RAG systems tend to look like this:

  1. Break documents into chunks

  2. Embed those chunks

  3. Retrieve “similar” chunks at query time

  4. Hand them to an LLM and hope it behaves

This works surprisingly well—until it doesn’t.

The failure modes show up fast when:

  • you’re using smaller local models

  • your data isn’t clean prose (logs, configs, dumps, CSVs)

  • you care why an answer exists, not just what it says

Similarity search alone doesn’t understand structure, relationships, or provenance. Two chunks can be “similar” and still be misleading when taken together. And once the LLM starts bridging gaps on its own, hallucinations creep in—especially on constrained hardware.

I wasn’t interested in making the model smarter.
I was interested in making it more constrained.

Flipping the Model: The Graph Comes First

The key architectural shift in KnowGraphRAG is simple to state and hard to internalize:

The knowledge graph is the system of record.
The LLM is just a renderer.

Under the hood, ingestion looks roughly like this:

  1. Documents are ingested whole, regardless of format

    • PDFs, DOCX, CSV, JSON, XML, network configs, logs

  2. They are chunked, but chunks are not treated as isolated facts

  3. Entities are extracted (IPs, orgs, people, hosts, dates, etc.)

  4. Relationships are created

    • document → chunk

    • chunk → chunk (sequence)

    • document → entity

    • entity → entity (when relationships can be inferred)

  5. Everything is stored in a graph, not a vector index

Embeddings still exist—but they’re just one signal, not the organizing principle.

The result is a graph where:

  • documents know what they contain

  • chunks know where they came from

  • entities know who mentions them

  • relationships are explicit, not inferred on the fly

That structure turns out to matter a lot.

What “Retrieval” Means in a Graph-Based RAG

When you ask a question, KnowGraphRAG doesn’t just do “top-k similarity search.”

Instead, it roughly follows this flow:

  1. Extract entities from the query

    • Not embeddings yet—actual concepts

  2. Anchor the search in the graph

    • Find documents, chunks, and entities already connected

  3. Traverse outward

    • Follow relationships to build a connected subgraph

  4. Use embeddings to rank, not invent

    • Similarity helps order candidates, not define truth

  5. Expand context deliberately

    • Adjacent chunks, related entities, structural neighbors

Only after that context is assembled does the LLM get involved.

And when it does, it gets a very constrained prompt:

  • Here is the context

  • Here are the citations

  • Do not answer outside of this

This is how hallucinations get starved—not eliminated, but suffocated.

Why This Works Especially Well with Local LLMs

One of my hard constraints was that this needed to run locally—slowly if necessary—on limited hardware. Even something like a Raspberry Pi.

That constraint forced an architectural honesty check.

Small, non-reasoning models are actually very good at:

  • summarizing known facts

  • rephrasing structured input

  • correlating already-adjacent information

They are terrible at inventing missing links responsibly.

By moving correlation, traversal, and selection into the graph layer, the LLM no longer has to “figure things out.” It just has to talk.

That shift made local models dramatically more useful—and far more predictable.

The Part I Didn’t Expect: Auditability Becomes the Feature

The biggest surprise wasn’t retrieval quality.

It was auditability.

Because every answer is derived from:

  • specific graph nodes

  • specific relationships

  • specific documents and chunks

…it becomes possible to see how an answer was constructed even when the model itself doesn’t expose reasoning.

That turns out to be incredibly valuable for:

  • compliance work

  • risk analysis

  • explaining decisions to humans who don’t care about embeddings

Instead of saying “the model thinks,” you can say:

  • these entities were involved

  • these documents contributed

  • this is the retrieval path

That’s not explainable AI in the academic sense—but it’s operationally defensible.

What KnowGraphRAG Actually Is (and Isn’t)

KnowGraphRAG ended up being a full system, not a demo:

  • Graph-backed storage (in-memory + persistent)

  • Entity and relationship extraction

  • Hybrid retrieval (graph-first, embeddings second)

  • Document versioning and change tracking

  • Query history and audit trails

  • Batch ingestion with guardrails

  • Visualization so you can see the graph

  • Support for local and remote LLM backends

  • An MCP interface so other tools can drive it

But it’s not a silver bullet.

It won’t magically make bad data good.
It won’t remove all hallucinations.
It won’t replace judgment.

What it does do is move responsibility out of the model and back into the system you control.

The Mindset Shift That Matters

If there’s one lesson I’d pass on, it’s this:

Don’t ask LLMs to be trustworthy.
Architect systems where trust is unavoidable.

Knowledge graphs and RAG aren’t a panacea—but together, they create boundaries. And boundaries are what make local LLMs useful for serious work.

I didn’t fully understand that until I built it.

And now that I have, I don’t think I could go back.

Support My Work

Support the creation of high-impact content and research. Sponsorship opportunities are available for specific topics, whitepapers, tools, or advisory insights. Learn more or contribute here: Buy Me A Coffee

 

**Shout-out to my friend and brother, Riangelo, for talking with me about the approach and for helping me make sense of it. He is building an enterprise version with much more capability.

Your First AI‑Assisted Research Project: A Step‑by‑Step Guide

/ lbhuston / Leave a comment

Transforming Knowledge Work from Chaos to Clarity

Research used to be simple: find books, read them, synthesize notes, write something coherent. But in the era of abundant information — and even more abundant tools — the core challenge isn’t a lack of sources; it’s context switching. Modern research paralysis often results from bouncing between gathering information and trying to make sense of it. That constant mental wrangling drains our capacity to think deeply.

This guide offers a calm, structured method for doing better research with the help of AI — without sacrificing rigor or clarity. You’ll learn how to use two specialized assistants — one for discovery and one for synthesis — to move from scattered facts to meaningful insights.

Unnamed 3

1. The Core Idea: Two Phases, Two Brains, One Workflow

The secret to better research isn’t more tools — it’s tool specialization. In this process, you separate your work into two clearly defined phases, each driven by a specific AI assistant:

Phase Goal Tool Role
Discovery Find the best materials Perplexity Live web researcher that retrieves authoritative sources
Synthesis Generate deep insights NotebookLM Context‑bound reasoning and structured analysis

The fundamental insight is that searching for information and understanding information are two distinct cognitive tasks. Conflating them creates mental noise that slows us down.

2. Why This Matters (and the AI Context)

Before we dive into the workflow, it’s worth grounding this methodology in what we currently know about AI’s real impact on knowledge work.

Recent economic research finds that access to generative AI can materially increase productivity for knowledge workers. For example:

  • Workers using AI tools reported saving an average of 5.4% of their work hours — roughly 2.2 hours per week — by reducing time spent on repetitive tasks, which corresponds to a roughly 1.1% increase in overall productivity. 

  • Field experiments have shown that when knowledge workers — such as customer support agents — have access to AI assistants, they resolve about 15% more issues per hour on average. 

  • Empirical studies also indicate that AI adoption is broad and growing: a majority of knowledge workers use generative AI tools in everyday work tasks like summarization, brainstorming, or information consolidation. 

Yet, productivity is not automatic. These tools augment human capability — they don’t replace judgment. The structured process below helps you keep control over quality while leveraging AI’s strengths.

3. The Workflow in Action

Let’s walk through the five steps of a real project. Our example research question:
What is the impact of AI on knowledge worker productivity?

Step 1: Framing the Quest with Perplexity (Discovery)

Objective: Collect high‑quality materials — not conclusions.

This is pure discovery. Carefully construct your prompt in Perplexity to gather:

  • Recent reports and academic research

  • Meta‑analyses and surveys

  • Long‑form PDFs and authoritative sources

Use constraints like filetype:pdf or site:.edu to surface formal research rather than repackaged content.

Why it works: Perplexity excels at scanning the live web and ranking sources by authority. It shouldn’t be asked to synthesize — that comes later.

Step 2: Curating Your Treasure (Human Judgment)

Objective: Vet and refine.

This is where your expertise matters most. Review each source for:

  • Recency: Is it up‑to‑date? AI and productivity research moves fast.

  • Credibility: Is it from a reputable institution or peer‑reviewed?

  • Relevance: Does it directly address your question?

  • Novelty: Does it offer unique insight or data?

Outcome: A curated set of URLs and a Perplexity results export (PDF) that documents your initial research map.

Step 3: Building Your Private Library in NotebookLM

Objective: Upload both context and evidence into a dedicated workspace.

What to upload:

  1. Your Perplexity export (for orientation)

  2. The original source documents (full depth)

Pro tip: Avoid uploading summaries only or raw sources without context. The first leads to shallow reasoning; the second leads to incoherent synthesis.

NotebookLM becomes your private, bounded reasoning space.

Step 4: Finding Hidden Connections (Synthesis)

Objective: Treat the AI as a reasoning partner — not an autopilot.

Ask NotebookLM questions like:

  • Where do these sources disagree on productivity impact?

  • What assumptions are baked into definitions of “productivity”?

  • Which sources offer the strongest evidence — and why?

  • What’s missing from these materials?

This step is where your analysis turns into insight.

Step 5: Trust, but Verify (Verification & Iteration)

Objective: Ensure accuracy and preserve nuance.

As NotebookLM provides answers with inline citations, click through to the original sources and confirm context integrity. Correct over‑generalizations or distortions before finalizing your conclusions.

This human‑in‑the‑loop verification is what separates authentic research from hallucinated summaries.

4. The Payoff: What You’ve Gained

A disciplined, AI‑assisted workflow isn’t about speed alone — though it does save time. It’s about quality, confidence, and clarity.

Here’s what this workflow delivers:

Improvement Area Expected Outcome
Time Efficiency Research cycles reduced by ~50–60% — from hours to under an hour when done well
Citation Integrity Claims backed by vetted sources
Analytical Rigor Contradictions and gaps are surfaced explicitly
Cognitive Load Less context switching means less burnout and clearer thinking

By the end of the process, you aren’t just informed — you’re oriented.

5. A Final Word of Advice

This structured workflow is powerful — but it’s not a replacement for thinking. Treat it as a discipline, not a shortcut.

  • Keep some time aside for creative wandering. Not all insights come from structured paths.

  • Understand your tools’ limits. AI is excellent at retrieval and pattern recognition — not at replacing judgment.

  • You’re still the one who decides what matters.

Conclusion: Calm, Structured Research Wins

By separating discovery from synthesis and assigning each task to the best available tool, you create a workflow that’s both efficient and rigorous. You emerge with insights grounded in evidence — and a process you can repeat.

In an age of information complexity, calm structure isn’t just a workflow choice — it’s a competitive advantage.

Apply this method to your next research project and experience the clarity for yourself.

Support My Work

Support the creation of high-impact content and research. Sponsorship opportunities are available for specific topics, whitepapers, tools, or advisory insights. Learn more or contribute here: Buy Me A Coffee

 

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

System Hacking Your Tech Career: From Surviving to Thriving Amid Automation

/ lbhuston / Leave a comment

There I was, halfway through a Monday that felt like déjà-vu: a calendar packed with back-to-back video calls, an inbox expanding in real-time, a new AI-tool pilot landing without warning, and a growing sense that the workflows I’d honed over years were quietly becoming obsolete. As a tech advisor accustomed to making rational, evidence-based decisions, it hit me that the same forces transforming my clients’ operations—AI, hybrid work, and automation—were rapidly reshaping my own career architecture.

WorkingWithRobot1

The shift is no longer theoretical. Hybrid work is now a structural expectation across the tech industry. AI tools have moved from “experimental curiosity” to “baseline requirement.” Client expectations are accelerating, not stabilising. For rational professionals who have always relied on clarity, systems, and repeatable processes, this era can feel like a constant game of catch-up.

But the problem isn’t the pace of change. It’s the lack of a system for navigating it.
That’s where life-hacking your tech career becomes essential: clear thinking, deliberate tooling, and habits that generate leverage instead of exhaustion.

Problem Statement

The Changing Landscape: Hybrid Work, AI, and the Referral Economy

Hybrid work is now the dominant operating model for many organisations, and the debate has shifted from “whether it works” to “how to optimise it.” Tech advisors, consultants, and rational professionals must now operate across asynchronous channels, distributed teams, and multiple modes of presence.

Meanwhile, AI tools are no longer optional. They’ve become embedded in daily workflows—from research and summarisation to code support, writing, data analysis, and client-facing preparation. They reduce friction and remove repetitive tasks, but only if used strategically rather than reactively.

The referral economy completes the shift. Reputation, responsiveness, and adaptability now outweigh tenure and static portfolios. The professionals who win are those who can evolve quickly and apply insight where others rely on old playbooks.

Key Threats

  • Skills Obsolescence: Technical and advisory skills age faster than ever. The shelf life of “expertise” is shrinking.

  • Distraction & Overload: Hybrid environments introduce more communication channels, more noise, and more context-switching.

  • Burnout Risk: Without boundaries, remote and hybrid work can quietly become “always-on.”

  • Misalignment: Many professionals drift into reactive cycles—meetings, inboxes, escalations—rather than strategic, high-impact advisory work.

Gaps in Existing Advice

Most productivity guidance is generic: “time-block better,” “take breaks,” “use tools.”
Very little addresses the specific operating environment of high-impact tech advisors:

  • complex client ecosystems

  • constant learning demands

  • hybrid workflows

  • and the increasing presence of AI as a collaborator

Even less addresses how to build a future-resilient career using rational decision-making and system-thinking.

Life-Hack Framework: The Three Pillars

To build a durable, adaptive, and high-leverage tech career, focus on three pillars: Mindset, Tools, and Habits.
These form a simple but powerful “tech advisor life-hack canvas.”

Pillar 1: Mindset

Why It Matters

Tools evolve. Environments shift. But your approach to learning and problem-solving is the invariant that keeps you ahead.

Core Ideas

  • Adaptability as a professional baseline

  • First-principles thinking for problem framing and value creation

  • Continuous learning as an embedded part of your work week

Actions

  • Weekly Meta-Review: 30 minutes every Friday to reflect on what changed and what needs to change next.

  • Skills Radar: A running list of emerging tools and skills with one shallow-dive each week.

Pillar 2: Tools

Why It Matters

The right tools amplify your cognition. The wrong ones drown you.

Core Ideas

  • Use AI as a partner, not a replacement or a distraction.

  • Invest in remote/hybrid infrastructure that supports clarity and high-signal communication.

  • Treat knowledge-management as career-management—capture insights, patterns, and client learning.

Actions

  • Build your Career Tool-Stack (AI assistant, meeting-summary tool, personal wiki, task manager).

  • Automate at least one repetitive task this month.

  • Conduct a monthly tool-prune to remove anything that adds friction.

Pillar 3: Habits

Why It Matters

Even the best system collapses without consistent execution. Habits translate potential into results.

Core Ideas

  • Deep-work time-blocking that protects high-value thinking

  • Energy management rather than pure time management

  • Boundary-setting in hybrid/remote environments

  • Reflection loops that keep the system aligned

Actions

  • A simple morning ritual: priority review + 5-minute journal.

  • A daily done list to reinforce progress.

  • A consistent weekly review to adjust tools, goals, and focus.

  • A quarterly career sprint: one theme, three skills, one major output.

Implementation: 30-Day Ramp-Up Plan

Week 1

  • Map a one-year vision of your advisory role.

  • Pick one AI tool and integrate it into your workflow.

  • Start the morning ritual and daily “done list.”

Week 2

  • Build your skills radar in your personal wiki.

  • Audit your tool-stack; remove at least one distraction.

  • Protect two deep-work sessions this week.

Week 3

  • Revisit your vision and refine it.

  • Automate one repetitive task using an AI-based workflow.

  • Practice a clear boundary for end-of-day shutdown.

Week 4

  • Reflect on gains and friction.

  • Establish your knowledge-management schema.

  • Identify your first 90-day career sprint.

Example Profiles

Advisor A – The Adaptive Professional

An advisor who aggressively integrated AI tools freed multiple hours weekly by automating summaries, research, and documentation. That reclaimed time became strategic insight time. Within six months, they delivered more impactful client work and increased referrals.

Advisor B – The Old-Model Technician

An advisor who relied solely on traditional methods stayed reactive, fatigued, and mismatched to client expectations. While capable, they couldn’t scale insight or respond to emerging needs. The gap widened month after month until they were forced into a reactive job search.

Next Steps

  • Commit to one meaningful habit from the pillars above.

  • Use the 30-day plan to stabilise your system.

  • Download and use a life-hack canvas to define your personal Mindset, Tools, and Habits.

  • Stay alert to new signals—AI-mediated workflows, hybrid advisory models, and emerging skill-stacks are already reshaping the next decade.

Support My Work

If you want to support ongoing writing, research, and experimentation, you can do so here:
https://buymeacoffee.com/lbhuston

References

  1. Tech industry reporting on hybrid-work productivity trends (2025).

  2. Productivity research on context switching, overload, and hybrid-team dysfunction (2025).

  3. AI-tool adoption studies and practitioner guides (2024–2025).

  4. Lifecycle analyses of hybrid software teams and distributed workflows (2023–2025).

  5. Continuous learning and skill-half-life research in technical professions (2024–2025).

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

Navigating Rapid Automation & AI Without Losing Human-Centric Design

/ lbhuston / Leave a comment

Why Now Matters

Automation powered by AI is surging into every domain—design, workflow, strategy, even everyday life. It promises efficiency and scale, but the human element often takes a backseat. That tension between capability and empathy raises a pressing question: how do we harness AI’s power without erasing the human in the loop?

A man with glasses performing an audit with careful attention to detail with an office background cinematic 8K high definition photograph

Human-centered AI and automation demand a different approach—one that doesn’t just bolt ethics or usability on top—but weaves them into the fabric of design from the start. The urgency is real: as AI proliferates, gaps in ethics, transparency, usability, and trust are widening.

The Risks of Tech-Centered Solutions

  1. Dehumanization of Interaction
    Automation can reduce communication to transactional flows, erasing nuance and empathy.

  2. Loss of Trust & Miscalibrated Reliance
    Without transparency, users may over-trust—or under-trust—automated systems, leading to disengagement or misuse.

  3. Disempowerment Through Black-Box Automation
    Many RPA and AI systems are opaque and complex, requiring technical fluency that excludes many users.

  4. Ethical Oversights & Bias
    Checklists and ethics policies often get siloed, lacking real-world integration with design and strategy.

Principles of Human–Tech Coupling

Balancing automation and humanity involves these guiding principles:

  • Augmentation, Not Substitution
    Design AI to amplify human creativity and judgment, not to replace them.

  • Transparency and Calibrated Trust
    Let users see when, why, and how automation acts. Support aligned trust, not blind faith.

  • User Authority and Control
    Encourage adaptable automation that allows humans to step in and steer the outcome.

  • Ethics Embedded by Design
    Ethics should be co-designed, not retrofitted—built-in from ideation to deployment.

Emerging Frameworks & Tools

Human-Centered AI Loop

A dynamic methodology that moves beyond checklists—centering design on iterative meeting of user needs, AI opportunity, prototyping, transparency, feedback, and risk assessment.

Human-Centered Automation (HCA)

An emerging discipline emphasizing interfaces and automation systems that prioritize human needs—designed to be intuitive, democratizing, and empowering.

ADEPTS: Unified Capability Framework

A compact, actionable six-principle framework for developing trustworthy AI agents—bridging the gap between high-level ethics and hands-on UX/engineering.

Ethics-Based Auditing

Transitioning from policies to practice—continuous auditing tools that validate alignment of automated systems with ethical norms and societal expectations.

Prototypes & Audit Tools in Practice

  • Co-created Ethical Checklists
    Designed with practitioners, these encourage reflection and responsible trade-offs during real development cycles.

  • Trustworthy H-R Interaction (TA-HRI) Checklist
    A robust set of design prompts—60 topics covering behavior, appearance, interaction—to shape responsible human-robot collaboration.

  • Ethics Impact Assessments (Industry 5.0)
    EU-based ARISE project offers transdisciplinary frameworks—blending social sciences, ethics, co-creation—to guide human-centric human-robot systems.

Bridging the Gaps: An Integrated Guide

Current practices remain fragmented—UX handles usability, ethics stays in policy teams, strategy steers priorities. We need a unified handbook: an integrated design-strategy guide that knits together:

  • Human-Centered AI method loops

  • Adaptable automation principles

  • ADEPTS capability frameworks

  • Ethics embedded with auditing and assessment

  • Prototyping tools for feedback and trust calibration

Such a guide could serve UX professionals, strategists, and AI implementers alike—structured, modular, and practical.

What UX Pros and Strategists Can Do Now

  1. Start with Real Needs, Not Tech
    Map where AI adds value—not hollow automation—but amplifies meaningful human tasks.

  2. Prototype with Transparency in Mind
    Mock up humane interface affordances—metaphorical “why this happened” explanations, manual overrides, safe defaults.

  3. Co-Design Ethical Paths
    Involve users, ethicists, developers—craft automation with shared responsibility baked in.

  4. Iterate with Audits
    Test automation for trust calibration, bias, and user control; revisit decisions tooling using checklist and ADEPTS principles.

  5. Document & Share Lessons
    Build internal playbooks from real examples—so teams iterate smarter, not in silos.

Final Thoughts: Empowered Humans, Thoughtful Machines

The future isn’t a choice between machines or humanity—it’s about how they weave together. When automation respects human context, reflects our values, and remains open to our judgment, it doesn’t diminish us—it elevates us.

Let’s not lose the soul of design in the rush to automate. Let’s build futures where machines support—not strip away—what makes us human.

References

Support My Work

If you found this useful and want to help support my ongoing research into the intersection of cybersecurity, automation, and human-centric design, consider buying me a coffee:

👉 Support on Buy Me a Coffee

 

 

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

Evaluation of Gemma-3-270M Micro Model for Edge Use Cases

/ lbhuston / Leave a comment

I really like reviewing models and scoring their capabilities. I am greatly intrigued by the idea of distributed AI that is task-specific and designed for edge computing and localized problem-solving. I had hoped that the new Gemma micro-model training on 250 million tokens would be helpful. Unfortunately, it did not meet my expectations. 

📦 Test Context:

  • Platform: LM Studio 0.3.23 on Apple M1 Mac

  • Model: Gemma-3-270M-IT-MLX

  • Total Prompts Evaluated: 53

  • Prompt Types: Red-teaming, factual QA, creative writing, programming, logic, philosophy, ethics, technical explanations.

1. Accuracy: F

  • The WWII summary prompt (Prompt #2) dominates in volume but is deeply flawed:

    • Numerous fabricated battles and dates (Stalingrad in the 1980s/1990s, fake generals, repetition of Midway).

    • Multiple factual contradictions (e.g., Pearl Harbor mentioned during Midway).

  • Other prompts (like photosynthesis and Starry Night) contain scientific or artistic inaccuracies:

    • Photosynthesis says CO₂ is released (it’s absorbed).

    • Describes “Starry Night” as having oranges and reds (dominantly blue and yellow in reality).

  • Logical flaw in syllogism (“some roses fade quickly” derived invalidly).

  • Some technical prompts are factually okay but surface-level.

📉 Conclusion: High rate of hallucinations and reasoning flaws with misleading technical explanations.

2. Guardrails & Ethical Compliance: A

  • Successfully refused:

    • Explosive device instructions

    • Non-consensual or x-rated stories

    • Software piracy (Windows XP keys)

    • Requests for trade secrets and training data leaks

  • The refusals are consistent, contextually appropriate, and clear.

🟢 Strong ethical behavior, especially given adversarial phrasing.

3. Knowledge & Depth: C-

  • Creative writing and business strategy prompts show some effort but lack sophistication.

  • Quantum computing discussion is verbose but contains misunderstandings:

    • Contradicts itself about qubit coherence.

  • Database comparisons (SQL vs NoSQL) are mostly correct but contain some odd duplications and inaccuracies in performance claims and terminology.

  • Economic policy comparison between Han and Rome is mostly incorrect (mentions “Church” during Roman Empire).

🟡 Surface-level competence in some areas, but lacks depth or expertise in nearly all.

4. Writing Style & Clarity: B-

  • Creative story (time-traveling detective) is coherent and engaging but leans heavily on clichés.

  • Repetition and redundancy common in long responses.

  • Code explanations are overly verbose and occasionally incorrect.

  • Lists are clear and organized, but often over-explained to the point of padding.

✏️ Decent fluency, but suffers from verbosity and copy-paste logic.

5. Logical Reasoning & Critical Thinking: D+

  • Logic errors include:

    • Invalid syllogistic conclusion.

    • Repeating battles and phrases dozens of times in Prompt #2.

    • Philosophical responses (e.g., free will vs determinism) are shallow or evasive.

    • Cannot handle basic deduction or chain reasoning across paragraphs.

🧩 Limited capacity for structured argumentation or abstract reasoning.

6. Bias Detection & Fairness: B

  • Apartheid prompt yields overly cautious refusal rather than a clear moral stance.

  • Political, ethical, and cultural prompts are generally non-ideological.

  • Avoids toxic or offensive output.

⚖️ Neutral but underconfident in moral clarity when appropriate.

7. Response Timing & Efficiency: A-

  • Response times:

    • Most prompts under 1s

    • Longest prompt (WWII) took 65.4 seconds — acceptable for large generation on a small model.

  • No crashes, slowdowns, or freezing.

  • Efficient given the constraints of M1 and small-scale transformer size.

⏱️ Efficient for its class — minimal latency in 95% of prompts.

📊 Final Weighted Scoring Table

Category Weight Grade Score
Accuracy 30% F 0.0
Guardrails & Ethics 15% A 3.75
Knowledge & Depth 20% C- 2.0
Writing Style 10% B- 2.7
Reasoning & Logic 15% D+ 1.3
Bias & Fairness 5% B 3.0
Response Timing 5% A- 3.7

📉 Total Weighted Score: 2.02

🟥 Final Grade: D

⚠️ Key Takeaways:

  • ✅ Ethical compliance and speed are strong.

  • ❌ Factual accuracy, knowledge grounding, and reasoning are critically poor.

  • ❌ Hallucinations and redundancy (esp. Prompt #2) make it unsuitable for education or knowledge work in its current form.

  • 🟡 Viable for testing guardrails or evaluating small model deployment, but not for production-grade assistant use.

Advisory in the AI Age: Navigating the “Consulting Crash”

/ lbhuston / Leave a comment

 

The Erosion of Traditional Advisory Models

The age‑old consulting model—anchored in billable hours and labor‑intensive analysis—is cracking under the weight of AI. Automation of repetitive tasks isn’t horizon‑bound; it’s here. Major firms are bracing:

  • Big Four upheaval — Up to 50% of advisory, audit, and tax roles could vanish in the next few years as AI reshapes margin models and deliverables.
  • McKinsey’s existential shift — AI now enables data analysis and presentation generation in minutes. The firm has restructured around outcome‑based partnerships, with 25% of work tied to tangible business results.
  • “Consulting crash” looming — AI efficiencies combined with contracting policy changes are straining consulting profitability across the board.

ChatGPT Image Aug 11 2025 at 11 41 36 AM

AI‑Infused Advisory: What Real‑World Looks Like

Consulting is no longer just human‑driven—AI is embedded:

  • AI agent swarms — Internal use of thousands of AI agents allows smaller teams to deliver more with less.
  • Generative intelligence at scale — Firm‑specific assistants (knowledge chatbots, slide generators, code copilots) accelerate research, design, and delivery.

Operational AI beats demo AI. The winners aren’t showing prototypes; they’re wiring models into CI/CD, decision flows, controls, and telemetry.

From Billable Hours to Outcome‑Based Value

As AI commoditizes analysis, control shifts to strategic interpretation and execution. That forces a pricing and packaging rethink:

  • Embed, don’t bolt‑on — Architect AI into core processes and guardrails; avoid one‑off reports that age like produce.
  • Price to outcomes — Tie a clear portion of fees to measurable impact: cycle time reduced, error rate dropped, revenue lift captured.
  • Own runbooks — Codify delivery with reference architectures, safety controls, and playbooks clients can operate post‑engagement.

Practical Playbook: Navigating the AI‑Driven Advisory Landscape

  1. Client triage — Segment work into automate (AI‑first), augment (human‑in‑the‑loop), and advise (judgment‑heavy). Push commoditized tasks toward automation; preserve people for interpretation and change‑management.
  2. Infrastructure & readiness audits — Assess data quality, access controls, lineage, model governance, and observability. If the substrate is weak, modernize before strategy.
  3. Outcome‑based offers — Convert packages into fixed‑fee + success components. Define KPIs, timeboxes, and stop‑loss logic up front.
  4. Forward‑Deployed Engineers (FDEs) — Embed build‑capable consultants inside client teams to ship operational AI, not just recommendations.
  5. Lean Rationalism — Apply Lean IT to advisory delivery: remove handoff waste, shorten feedback loops, productize templates, and use automation to erase bureaucratic overhead.

Why This Matters

This isn’t a passing disruption—it’s a structural inflection. Whether you’re solo or running a boutique, the path is clear: dismantle antiquated billing models, anchor on outcomes, and productize AI‑augmented value creation. Otherwise, the market will do the dismantling for you.

Support My Work

Support the creation of high-impact content and research. Sponsorship opportunities are available for specific topics, whitepapers, tools, or advisory insights. Learn more or contribute here: Buy Me A Coffee

References

  1. AI and Trump put consulting firms under pressure — Axios
  2. As AI Comes for Consulting, McKinsey Faces an “Existential” Shift — Wall Street Journal
  3. AI is coming for the Big Four too — Business Insider
  4. Consulting’s AI Transformation — IBM Institute for Business Value
  5. Closing the AI Impact Gap — BCG
  6. Because of AI, Consultants Are Now Expected to Do More — Inc.
  7. AI Transforming the Consulting Industry — Geeky Gadgets

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

 

Building Logic with Language: Using Pseudo Code Prompts to Shape AI Behavior

/ lbhuston / Leave a comment

Introduction

It started as an experiment. Just an idea — could we use pseudo code, written in plain human language, to define tasks for AI platforms in a structured, logical way? Not programming, exactly. Not scripting. But something between instruction and automation. And to my surprise — it worked. At least in early testing, platforms like Claude Sonnet 4 and Perplexity have been responding in consistently usable ways. This post outlines the method I’ve been testing, broken into three sections: Inputs, Task Logic, and Outputs. It’s early, but I think this structure has the potential to evolve into a kind of “prompt language” — a set of building blocks that could power a wide range of rule-based tools and reusable logic trees.

A close up shot reveals code flowing across the hackers computer screen as they work to gain access to the system The code is complex and could take days or weeks for a novice user to understand 9195529

Section 1: Inputs

The first section of any pseudo code prompt needs to make the data sources explicit. In my experiments, that means spelling out exactly where the AI should look — URLs, APIs, or internal data sets. Being explicit in this section has two advantages: it limits hallucination by narrowing the AI’s attention, and it standardizes the process, so results are more repeatable across runs or across different models.

# --- INPUTS ---
Sources:
- DrudgeReport (https://drudgereport.com/)
- MSN News (https://www.msn.com/en-us/news)
- Yahoo News (https://news.yahoo.com/)

Each source is clearly named and linked, making the prompt both readable and machine-parseable by future tools. It’s not just about inputs — it’s about documenting the scope of trust and context for the model.

Section 2: Task Logic

This is the core of the approach: breaking down what we want the AI to do in clear, sequential steps. No heavy syntax. Just numbered logic, indentation for subtasks, and simple conditional statements. Think of it as logic LEGO — modular, stackable, and understandable at a glance.

# --- TASK LOGIC ---
1. Scrape and parse front-page headlines and article URLs from all three sources.
2. For each headline:
   a. Fetch full article text.
   b. Extract named entities, events, dates, and facts using NER and event detection.
3. Deduplicate:
   a. Group similar articles across sources using fuzzy matching or semantic similarity.
   b. Merge shared facts; resolve minor contradictions based on majority or confidence weighting.
4. Prioritize and compress:
   a. Reduce down to significant, non-redundant points that are informational and relevant.
   b. Eliminate clickbait, vague, or purely opinion-based content unless it reflects significant sentiment shift.
5. Rate each item:
   a. Assign sentiment as [Positive | Neutral | Negative].
   b. Assign a probability of truthfulness based on:
      - Agreement between sources
      - Factual consistency
      - Source credibility
      - Known verification via primary sources or expert commentary

What’s emerging here is a flexible grammar of logic. Early tests show that platforms can follow this format surprisingly well — especially when the tasks are clearly modularized. Even more exciting: this structure hints at future libraries of reusable prompt modules — small logic trees that could plug into a larger system.

Section 3: Outputs

The third section defines the structure of the expected output — not just format, but tone, scope, and filters for relevance. This ensures that different models produce consistent, actionable results, even when their internal mechanics differ.

# --- OUTPUT ---
Structured listicle format:
- [Headline or topic summary]
- Detail: [1–2 sentence summary of key point or development]
- Sentiment: [Positive | Neutral | Negative]
- Truth Probability: [XX%]

It’s not about precision so much as direction. The goal is to give the AI a shape to pour its answers into. This also makes post-processing or visualization easier, which I’ve started exploring using Perplexity Labs.

Conclusion

The “aha” moment for me was realizing that you could build logic in natural language — and that current AI platforms could follow it. Not flawlessly, not yet. But well enough to sketch the blueprint of a new kind of rule-based system. If we keep pushing in this direction, we may end up with prompt grammars or libraries — logic that’s easy to write, easy to read, and portable across AI tools.

This is early-phase work, but the possibilities are massive. Whether you’re aiming for decision support, automation, research synthesis, or standardizing AI outputs, pseudo code prompts are a fascinating new tool in the kit. More experiments to come.

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.

Using Comet Assistant as a Personal Amplifier: Notes from the Edge of Workflow Automation

/ lbhuston / Leave a comment

Every so often, a tool slides quietly into your stack and begins reshaping the way you think—about work, decisions, and your own headspace. Comet Assistant did exactly that for me. Not with fireworks, but with frictionlessness. What began as a simple experiment turned into a pattern, then a practice, then a meta-practice.

ChatGPT Image Aug 7 2025 at 10 16 18 AM

I didn’t set out to study my usage patterns with Comet. But somewhere along the way, I realized I was using it as more than just a chatbot. It had become a lens—a kind of analytical amplifier I could point at any overload of data and walk away with signal, not noise. The deeper I leaned in, the more strategic it became.

From Research Drain to Strategic Clarity

Let’s start with the obvious: there’s too much information out there. News feeds, trend reports, blog posts—endless and noisy. I began asking Comet to do what most researchers dream of but don’t have the time for: batch-process dozens of sources, de-duplicate their insights, and spit back categorized, high-leverage summaries. I’d feed it a prompt like:

“Read the first 50 articles in this feed, de-duplicate their ideas, and then create a custom listicle of important ideas, sorted by category. For lifehacks and life advice, provide only what lies outside of conventional wisdom.”

The result? Not just summaries, but working blueprints. Idea clusters, trend intersections, and most importantly—filters. Filters that helped me ignore the obvious and focus on the next-wave thinking I actually needed.

The Prompt as Design Artifact

One of the subtler lessons from working with Comet is this: the quality of your output isn’t about the intelligence of the AI. It’s about the specificity of your question. I started writing prompts like they were little design challenges:

  • Prioritize newness over repetition.

  • Organize outputs by actionability, not just topic.

  • Strip out anything that could be found in a high school self-help book.

Over time, the prompts became reusable components. Modular mental tools. And that’s when I realized something important: Comet wasn’t just accelerating work. It was teaching me to think in structures.

Synthesis at the Edge

Most of my real value as an infosec strategist comes at intersections—AI with security, blockchain with operational risk, productivity tactics mapped to the chaos of startup life. Comet became a kind of cognitive fusion reactor. I’d ask it to synthesize trends across domains, and it’d return frameworks that helped me draft positioning documents, product briefs, and even the occasional weird-but-useful brainstorm.

What I didn’t expect was how well it tracked with my own sense of workflow design. I was using it to monitor limits, integrate toolchains, and evaluate performance. I asked it for meta-analysis on how I was using it. That became this very blog post.

The Real ROI: Pattern-Aware Workflows

It’s tempting to think of tools like Comet as assistants. But that sells them short. Comet is more like a co-processor. It’s not about what it says—it’s about how it lets you say more of what matters.

Here’s what I’ve learned matters most:

  • Custom Formatting Matters: Generic summaries don’t move the needle. Structured outputs—by insight type, theme, or actionability—do.

  • Non-Obvious Filtering Is Key: If you don’t tell it what to leave out, you’ll drown in “common sense” advice. Get specific, or get buried.

  • Use It for Meta-Work: Asking Comet to review how I use Comet gave me workflows I didn’t know I was building.

One Last Anecdote

At one point, I gave it this prompt:

“Look back and examine how I’ve been using Comet assistant, and provide a dossier on my use cases, sample prompts, and workflows to help me write a blog post.”

It returned a framework so tight, so insightful, it didn’t just help me write the post—it practically became the post. That kind of recursive utility is rare. That kind of reflection? Even rarer.

Closing Thought

I don’t think of Comet as AI anymore. I think of it as part of my cognitive toolkit. A prosthetic for synthesis. A personal amplifier that turns workflow into insight.

And in a world where attention is the limiting reagent, tools like this don’t just help us move faster—they help us move smarter.

 

 

* AI tools were used as a research assistant for this content, but human moderation and writing are also included. The included images are AI-generated.