Hello everyone!
I am back! I appreciate everyone who has been posting in the interim. Love to see new content being shared.
This thread is inspired by one submitted by @cll7793@lemmy.world, who in this post detailed general LLM resources for everyone to discover and interact with (content which has now been consolidated to the full LLM guide here, thanks cll!).
All that being said, I promised I would get back to them on a few questions they had. I figured why not make a thread dedicated to questions any of you would like to ask me.
AMA
If you are here and subscribed to !fosai@lemmy.world - please feel more than welcome asking your question in the comments below. I will try to get back to you within a day or two. I will do my best to answer all questions within reason.
Note that I am not an expert in Machine Learning, but I do work as a professional close to this category of tech. Take my responses with a grain of salt and cross-reference other sources to make sure you are informed.
Always do your own research on top of any of my advice. I am not responsible for the success or failure of your endeavor. Own up to your builds, own up to your development. I am but a resource and friend along the way. A humble guide on your journey to the singularity.
Questions
from @cll7793@lemmy.world
Q1.) What people do you follow for AI? Such as on YT, Twitter, etc.
A1.) I am not on social media often. Lemmy is my safe haven, alongside very specific YouTube channels who give me the information I seek.
Generally speaking, I try to avoid the sensationalism and stay focused on how I can empower myself with this tech now. So far, the channels below have helped me stay informed with practical insights and information.
Whatever it is you’re doing, be sure to have a goal in mind. You will spend an eternity learning with no direction.
For me, it was learning how to regularly and consistently train, fine-tune, and deploy open-source models on HuggingFace. That goal has coincidentally led to the creation of this community and my newfound passion for Machine Learning & AI.
Matthew Berman
I like Matthew. He’s cool. Always on top of AI news.
AemonAlgiz
Aemon is great, provides both general education and technical tutorials.
Aitrepreneur
Useful info, just wish the channel wasn’t so clickbaity. I get it though.
AI Jason
Great for more technical tutorials that you can follow step-by-step.
Venelin Valkov
Another source for tutorials.
Code Your Own AI
General education.
Q2.) What other social media forums provide great information?
A2.) Like I mentioned in my last response, I’m not on social media often. I’m a year one gen-z’er (1997) who never really got into social media (until now). Weird, I know. That being said, the only other place I frequent for knowledge is Lex Fridman’s Podcasts - which are treasure troves of information if you enjoy the discussions and/or content. Particularly any of the episodes with tech-based content.
Even if you don’t like some of the guests, you can infer a lot of interesting information based on what context they share with Lex. Much of what I have pieced together for our community here at !fosai@lemmy.world is information I have cross referenced with other independent papers, news, and articles I find credible. I think that’s called journalism, but the definition of that has been skewed compared to today’s social media standards.
Aside from those podcasts, I am a big fan of the /r/LocalLLaMA community, but Reddit is hard for me to revisit nowadays. It just doesn’t feel the same after the API change. I feel the best parts have left or are tucked in corners that are hard to find.
In other words, I get all my info from those YouTube channels I shared above, random posts and comments I find in niche corners, and/or published papers or articles from firsthand sources (typically Arxiv or Medium).
That, and reading documentation. Lots and lots of documentation on whatever it is I’m using at the time…
Q3.) What GUI do you use for local LLMs?
A3.) I might be biased because I’m a linux & vim kind of guy, so I am totally fine working in a terminal - but if I need a consistent GUI for testing, I typically default to oobabooga’s text-generation-webui.
When I am using a GUI that’s not oobabooga’s, it’s for R&D. In the past I have found myself testing with SillyTavern, koboldAI’s client, and gpt4all - but none of them stuck with my workflows. I revisit them from time-to-time to see how they’ve developed. They’re definitely improving and worth taking a look at if you haven’t already.
I am finding a lot of new interfaces to chat with LLMs, but not any in particular that stands out above the rest in terms of look, feel, and functionality. I think it’s only a matter of time before we see even more UI/UX’s tailored for LLMs. Until then, I’ll be using ooba and other side projects I tinker with from time to time.
When in doubt, you can always learn to code/design your own GUI, but that’s pretty advanced if you’re just starting to get into all of this. At the end of the day, you should use what’s best for you or what you prefer (or like) using the most.
Q4.) What parameters are “best”?
A4.) This is a tricky question. It could mean a lot of things, I’ll do my best.
a.) If this is in reference to hyper parameters and general parameters, I suggest reading this article for starters.
b.) If this question is in reference to what size parameters I personally find ‘best’ within LLMs (3B, 7B, 13B, etc), well, I’d say the best model is anything you can comfortably run that outputs the results you’re hoping for. That could mean a high parameter Falcon 40B model, or perhaps a low parameter 3B Nous model.
The results are so drastically different between use cases and performance that I say experiment with all and see which is the most consistent for you particularly.
For fellow technologists running LLMs at home, that probably means you’re running this on consumer-grade hardware. For desktop builds with NVIDIA GPUs I’ve found really consistent performance from 7B and 13B parameter models.
With some optimization, you can speed up inference depending on what base model you’re using and what platform you’re running it on. Sometimes even some of the 3B models surprise me with their coherence and perplexity. It’s up to you to find what works for you best.
c.) If you’re asking about specific parameters I apply during training - I don’t have an answer to that yet. I haven’t completed enough deployments to give you a worthy response.
Q5.) Is there a Wiki you use?
A5.) I have not found an open-source wiki that suits my need, so I made FOSAI ▲ XYZ to fill help in the gap and serve this community. It’s still a work in progress. I sort of used this to practice my web dev. I have plans on upgrading it to a format that will better serve the variety of content it presents. It’s a little dry IMO, but here nonetheless. I’ll be sure to let everyone know when the new site is live!
When in doubt, read the native documentation of the software (or hardware) you choose to use. It might be a dense read, but nothing CTRL + F can’t help with.
Q6.) Where do you go to learn about LLMs/AI/Machine Learning?
A6.) Like everything I do, I self teach myself concept by concept with a results-based mindset referencing all of the learning material and channels I shared a few questions ago.
I frequently fall victim to paralysis by analysis, so I combat this by dividing my ambition into actionable projects with the primary intention of getting them up and running (with reconciliation of notes, knowledge, and understanding after the fact).
I think it’s important to adopt a mindset that is brave to jump right into the middle of something without understanding any of it. Almost like exposure therapy.
Over time - concepts will become less alien and things will begin to click. Consistency is key. In my experience, curiosity empowers wonder, which can lead to motivation to learn something you care about. If you care about Machine Learning as much as I do, you’ll find ways to teach yourself through sheer will.
On my quest to better understand Machine Learning, I put together an online syllabus that is supposed to equip you with a core foundation of skills necessary to become a professional in the field. This is pieced together from a few videos I stumbled across and recommendations from colleagues in the field. Granted, I have the advantage of already being in tech, but I hold no degree - just a high school diploma.
Math is hard, but part of the fun once you realize the fact that it’s the language of reality, physics, computers and science. If you want to learn the same material I am, check out the Machine Learning page from FOSAI ▲ XYZ. Pick and choose what you want to learn. It is non-linear, non-sequential, and here to be learned at your own pace.
Q7.) How do you find quality models?
A7.) Every other day I check and see what TheBloke has released on his HuggingFace. As far as I’m concerned, he has become an open-source hero for the field. I trust everything he puts out, often cherry picking the model I want to test based on their origin card or foundation model.
If there’s a model worth running, Tom is fast to optimize a GPTQ or GGML model that he hosts on his page for all of us to interface with (for free). A true gigachad.
Support Tom on Patreon! Some his tiers offer direct technical support if you’re hitting a wall and need some hands-on help: https://www.patreon.com/TheBlokeAI
Q8.) What Awesome github repositories do you know?
A8.) I’ve been waiting for someone to ask me this! Haha. Finally…
I have been collecting all kinds of interesting repo’s for FOSS & FOSAI technologies. I am planning to do a new series soon where I dive into each one in more detail.
If you want a head start on that series, check out all of the cool tech people are developing here: https://github.com/adynblaed?tab=stars
I will be highlighting a few for AI / LLM particular use cases soon. You and everyone else here are welcome to explore some of the stars I’ve collected in the meantime.
Q9.) What do you think would be useful to share?
A9.) For !fosai@lemmy.world, I think it’s useful for everyone to share their experiences with using FOSS & FOSAI platforms (anything listed in our sidebar) from performance to discoveries, breakthroughs and celebrations.
To be more specific, it would be awesome to see how everyone is using LLMs on a day-to-day basis (if at all), and how it has impacted their lives productively and/or recreationally. For me, this has manifested in the fact I use Google Search 20% of the time I used to, since local LLMs + ChatGPT have replaced primary ‘search’ for me. This is a big deal because I work in tech and I can’t tell you how much of my job was having decent Google-Fu. That has changed into how well I can prompt and how effectively I can extract usable information from a generative pre-trained transformer.
Now that this tech is here, spreading, and open-sourced - I feel we’re in the calm before the storm - the crest of the rising tsunami of change the world is about to see in the next few years. I am excited to brave this ship and ride the crest with each and every one of you - but knowing where you want this ship to sail gives me insights that helps me adjust the content I provide (however long I can provide it) and course correct as needed.
Everyone posting their experience with how AI / LLMs / free open-source artificial intelligence are impacting them now, today is absolutely worth sharing even if it seems small or insignificant. I use this community as a time capsule as much as I do engaging with it for learning, sharing, and staying in the know.
Looking back at what we’re talking about in 2023 versus what we might be talking about in 2024 will be an interesting context to study, especially as the world begins to adopt AI.
Sharing ideas can lead to discussions, with enough reinforcement these ideas become concepts.
Concepts have opportunity to turn to projects, which develop into realities for all of us to explore together.
Whatever your idea is, please share it. No matter how large or small, silly or ridiculous. If it’s satire, label it and let’s have fun in the absurdity. If it’s legitimate, let’s explore the possibilities and draw collective curiosity towards a common goal. This is how we grow and empower one another.
To innovate is to think, to think is to live.
Share your innovations! !fosai@lemmy.world is a safe space to create, think, express, invent, and explore the mind, the self, and the machine.
Quick info: We have !localllama@sh.itjust.works over here. But I don’t think we’ve had an coordinated effort (yet) to move away from Reddit, so it’s only a few people. (I bet people also split up and some moved to some random discord communities. Nowadays many discussions happen behind closed doors scattered over small and larger discord communities. And LocalLlama on reddit is also still pretty active.)
I will say, if I’m on Reddit, it’s for /r/LocalLLaMA. For the most part, everyone there has been very helpful and supportive of the tech. Glad to see parts of it on Lemmy! Thanks for sharing, I’m going to add it to the community page on FOSAI ▲ XYZ.