Posts

Discovering English syntax

I've started a new project. My goal is to write a program that discovers enough of the syntax rules of written English to serve as a passable lexical tokenizer. I've made some progress in my approach thus far. But I can tell that my approach requires some serious rethinking. I'll describe the experimental design here and comment on my current progress. If you wish to see the code I'm actively experimenting with you can find it on GitHub . English syntax Anyone familiar with programming language will recognize that there is a process involved in translating human-readable code into the more cryptic representation used by a computer to execute that code. And that there is a very precise syntax governing what your code in that language must look like to be considered syntactically valid. In the JavaScript statement "var x = someFunction(y * 12);" you'll realize that there are well-defined roles for each part. The "var" keyword indicates that the &qu

Neural network in C# with multicore parallelization / MNIST digits demo

Image
I've been working for a couples weeks on building my first fully functional artificial neural network (ANN). I'm not blazing any news trails here by doing so. I'm a software engineer. I can barely follow the mathematical explanations of how ANNs work. For the most part I have turned to the source code others have shared online for inspiration. In most cases I've struggled to understand even that, despite programming for a living. Part of the challenge is that more than a few of those demos have surprised me by being nonfunctional. They did something for sure. They just didn't learn anything or perform significantly better than random chance making correct predictions, no matter how many iterations they went through. Or they had bugs that prevented them from working according to the well-worn basic backpropagation algorithm. I mostly worked from C# examples when I could find them. One thing that was a genuine struggle for me to deal with is my sense that they all

Back in the saddle / C# neural network demo

I felt like I was making good progress back in 2016 in my AI research. But I realized it was not going to turn into an income-yielding venture anytime soon. So I moved on to other ventures. This was during a sabbatical from "real work". I eventually gave up and got a real job again. While that job pays reasonably well, it's also fairly boring. Kira and I moved from Madison to Miami this year. For various reasons I decided to do a YouTube channel for a while to give my impressions of life here. That was fun for a while Recently I got somewhat bored with that and put it on hold. I was in a bit of a funk about what I should do with my free time when I'm not doing my regular work. I decided to resume my AI research. I've been dipping my toes in this week. Reading back on my own blog to reconnect with my most recent project. I decided to start into the subject of artificial neural networks (ANNs) again. I'm a little embarrassed to admit that I never wrote a traditi

Virtual lexicon vs Brown corpus

Having completed my blocker , I decided to take a break before tackling syntax analysis to study more facets of English. But also, I realized I should beef up the lexicon underlying my virtual lexicon (VL). I had only collected about 1,500 words, and most of those I had simply hand-entered by way of theft from the CGEL's chapters on morphology; mostly compound words, at that. It was enough to test and demonstrate the VL's capacity to deal half-decently with morphological parsing, but nowhere near big enough to represent the at least tens of thousands of words a typical high school graduate with English as their native language will know. A virtual lexicon's core premise is that being able to recognize novel word forms by recognizing the parts of the word is more valuable than having a large list of exacting word-forms. In essence, a relatively small number of lexical entries should be able to represent a much larger set of practical words found "in the wild".