Posts

Praxworx - Subjects lab

By
On August 27th I started planning out a new AI research project that I now call the "Subjects lab". The purpose was to build a better memory subsystem for today's LLM-powered conversational agents (CAs). One essential premise was that the best memory a CA can have is all of its earlier conversations with the current user or many of them within an organization. Since today's LLMs have limited context windows (active memory), they need a way to be aware of their whole conversation history without having to load all of it into context. The heart of this mechanism is an associative memory that indexes conversation history on subjects the CA identifies and manages. As a conversation develops, one extra LLM is mining it for a list of subjects that the user covers. The process creates something analogous to an old subject-centric library card catalogue. So as the user engages with the CA via prompts, those subjects get recognized very quickly along with shorthand summaries...
Post a Comment
Read more

My Praxworx venture

By
Since August 28th of this year (2026) I have been working quietly on a new AI-centric venture. It's best to think of it as an R&D project more than a company per se. For now. I started out with a goal to pursue something I've dubbed "artificial expertise". I'll discuss that in a forthcoming post. But from the start I have taken the stance that I should build one "lab" after another exploring different aspects of AI, mostly around a larger goal of cobbling together better behaviors than even the best that AI tech companies have to offer today. One thing I must confess about this process is that the AI companies are moving incredibly quickly. I'll start working on something I think is very innovative. Then I'll start seeing commercially viable features getting rolled out. Sometimes with great fanfare. Sometimes as quiet upgrades to existing apps. It's intimidating working at the cutting edge in both cooperation and yet competition with comp...
Post a Comment
Read more

Coherence and ambiguities in problem solving

By
Natural Language Processing (NLP) is a big topic. One I come back to again and again when I have time to explore it. Work has kept me very busy. So has moving. In recent months I've returned to the topic. I made some interesting progress constructing an "NLP pipeline". But as anyone who has done NLP work will tell you, English is full of ambiguities. They may tell you about the approaches they take to reduce the ambiguity and be decisive in the end. But the ambiguities persist and can't simply be guessed away. The problem More importantly though. The ambiguities often require us as humans to look across levels of interpretation to resolve. I generally have not found AI researchers offering a good way of doing this though. To illustrate my problem, consider this sentence: Some guys' shoes' laces are red. As someone literate in English you have no problem interpreting it. But odds are good you can guess where the ambiguity lies. Looked at in isolation, each of t...
1 comment
Read more

Discovering English syntax

By
I've started a new project. My goal is to write a program that discovers enough of the syntax rules of written English to serve as a passable lexical tokenizer. I've made some progress in my approach thus far. But I can tell that my approach requires some serious rethinking. I'll describe the experimental design here and comment on my current progress. If you wish to see the code I'm actively experimenting with you can find it on GitHub . English syntax Anyone familiar with programming language will recognize that there is a process involved in translating human-readable code into the more cryptic representation used by a computer to execute that code. And that there is a very precise syntax governing what your code in that language must look like to be considered syntactically valid. In the JavaScript statement "var x = someFunction(y * 12);" you'll realize that there are well-defined roles for each part. The "var" keyword indicates that the ...
1 comment
Read more

Neural network in C# with multicore parallelization / MNIST digits demo

By
Image
I've been working for a couple weeks on building my first fully functional artificial neural network (ANN). I'm not blazing any news trails here by doing so. I'm a software engineer. I can barely follow the mathematical explanations of how ANNs work. For the most part I have turned to the source code others have shared online for inspiration. In most cases I've struggled to understand even that, despite programming for a living. Part of the challenge is that more than a few of those demos have surprised me by being nonfunctional. They did something for sure. They just didn't learn anything or perform significantly better than random chance making correct predictions, no matter how many iterations they went through. Or they had bugs that prevented them from working according to the well-worn basic backpropagation algorithm. I mostly worked from C# examples when I could find them. One thing that was a genuine struggle for me to deal with is my sense that they all ...
1 comment
Read more

Back in the saddle / C# neural network demo

By
I felt like I was making good progress back in 2016 in my AI research. But I realized it was not going to turn into an income-yielding venture anytime soon. So I moved on to other ventures. This was during a sabbatical from "real work". I eventually gave up and got a real job again. While that job pays reasonably well, it's also fairly boring. Kira and I moved from Madison to Miami this year. For various reasons I decided to do a YouTube channel for a while to give my impressions of life here. That was fun for a while Recently I got somewhat bored with that and put it on hold. I was in a bit of a funk about what I should do with my free time when I'm not doing my regular work. I decided to resume my AI research. I've been dipping my toes in this week. Reading back on my own blog to reconnect with my most recent project. I decided to start into the subject of artificial neural networks (ANNs) again. I'm a little embarrassed to admit that I never wrote a traditi...
Post a Comment
Read more