• 0 Posts
  • 3 Comments
Joined 2 years ago
cake
Cake day: June 17th, 2023

help-circle
  • I’m pretty sure that phone keyboard use heuristics and not Machine Learning. Basically, it does not create a neural network through trial and error, but whenever you type, it saves the context of each word, and when it sees the same context again, it “knows” what the next word is.

    For example, if you type this big brown fox, it might saves something like "{ fox", ["big", "brown"], 1 } (assuming two words of context, and the 1 being the number of times it was encountered). Then when you type my big brown, fox will be suggested.

    Using the technology of LLMs for keyboard suggestions is impractical, as your typing habits would be drowned in the initial training data, and would yield worse performance as well as results compared to the simpler approach.


  • Having worked with ML in manufacturing, if your task is precise enough and your input normalized enough, it can detect very impressive things. Identifying mushrooms as a whole is already too grand a task, especially as it as to deal with different camera angles, lighting … But ask it to differentiate between a few species, and always offer pictures using similar angles, lighting and background, and the results will most likely be stellar.


  • What I think is missing from your viewpoint (and from most people’s, this is [IMO] a problem at scale) is the distinction between “simple” and broad machine learning, and the very specific things that are Large Language Models.

    For example, there are no small Large Language Models, and I think that the oxymoron speaks for itself. Machine learning is a very good thing, and automated classification is definitely its best use case, but they are not a small version of ChatGPT, the same way that the average Joe is not a smaller version of a billionaire.

    For more details, these small models are trained on a small set of data, how small depending on how specific the task is; as an example, I worked with models that detect manufacturing defects on production lines, and theses need a few hundreds images in order to produce good results, this make it very easy to produce the data ourselves, and it is relatively cheap to train energy-wise.

    Compared to that, Large Language Models, and their audiovisual counterparts, operate on billions of data, and work on a task so general that they provide incredibly bad results. As a little statistical reminder, anything below 95% confidence is a bust, LLMs are way below that.

    It’s very important to distinguish the two, because all of the positives you list for AI are not about LLMs, but about simple machine learning. And this confusion is by design, techbros are trying to profit of the successes of other form of artificial intelligence by pretending that AI is this one single thing, instead of an entire class of things.

    Otherwise, I generally agree with the rest of your points.