LoRA is one of the most widely used, parameter-efficient finetuning techniques for training custom LLMs. From saving memory with QLoRA to selecting the optimal LoRA settings, this article provides practical insights for those interested in applying it.
As a machine learning engineer who has witnessed the rise of Large Language Models (LLMs), I find it daunting to comprehend how the ecosystem surrounding LLMs is developing.
In this guide, we will learn how to develop and productionize a retrieval augmented generation (RAG) based LLM application, with a focus on scale and evaluation.
Hey everyone! I’ve mostly started sharing my favorite memos and transcripts over at A Letter a Day, but as it’s more of a tasting menu, I’m reticent to post multiple memos or tran…
When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically...
Data Curation, Transformers, Training at Scale, and Model Evaluation
I want to provide some tips from my experience implementing a paper. I'm going to cover my tips so far from implementing a dramatically scaled-down versio...
Aman's AI Journal | Course notes and learning material for Artificial Intelligence and Deep Learning Stanford classes.
How to fine-tune Llama and other LLMs with one tool
A multifaceted challenge has arisen in the expansive realm of natural language processing: the ability to adeptly comprehend and respond to intricate and lengthy instructions. As communication nuances become more complicated, the shortcomings of prevailing models in dealing with…
Recent language models can take long contexts as input; more is needed to know about how well they use longer contexts. Can LLMs be extended to longer contexts? This is an unanswered question. Researchers at Abacus AI conducted multiple experiments involving different schemes…
Using ChatGPT & OpenAI's GPT API, this code tutorial teaches how to chat with PDFs, automate PDF tasks, and build PDF chatbots.
Practical Advice from Experts: Fine-Tuning, Deployment, and Best Practices
My town, once celebrated for its laid-back weirdness, is now a turbocharged tech megalopolis being shaped by exiles from places like Silicon Valley.
Navigating the maze of pricing plans for digital services can sometimes be a daunting task. Today, we are unveiling Midjourney
Exploring the Development of the 3 Leading Open LLMs and Their Chatbot Derivatives
A reference architecture for the LLM app stack. It shows the most common systems, tools, and design patterns used by AI startups and tech companies.
Large language models such as GPT-3/4, LLaMA and PaLM work in terms of tokens. They take text, convert it into tokens (integers), then predict which tokens should come next. Playing …
tldr; techniques to speed up training and inference of LLMs to use large context window up to 100K input tokens during training and…
This article provides a series of techniques that can lower memory consumption in PyTorch (when training vision transformers and LLMs) by approximately 20x without sacrificing modeling performance and prediction accuracy.
It crazy how far the ML field has come when it comes to fine-tuning LLMs. A year ago: it was challenging to fine-tune GPT-2 (1.5B) on a single GPU without… | 76 comments on LinkedIn
A comprehensive guide on how to use Meta's LLaMA 2, the new open-source AI model challenging OpenAI's ChatGPT and Google's Bard.
We learn a lot of lessons from the hyperscalers and HPC centers of the world, and one of them is that those who control their own software control their
How LLaMA is making open-source cool again
Not only has LLaMA been trained on more data, with more parameters, the model also performs better than its predecessor, according to Meta.
MosaicML claims that the MPT-7B-8K LLM exhibits exceptional proficiency in summarization and answering tasks compared to previous models.
A curated list of practical guide resources of LLMs (LLMs Tree, Examples, Papers) - Mooler0410/LLMsPracticalGuide
Falcon LLM, is the new large language model that has taken the crown from LLaMA.
Get started using Falcon-7B, Falcon-40B, and their instruct versions
Seamlessly integrate LLMs into scikit-learn.
GPUs may dominate, but CPUs could be perfect for smaller AI models
AI is getting very chatty! Here’s a visualisation charting the rise of Large Language Models like GPT4, LaMDA, LLaMa, PaLM and their bots...
OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset - openlm-research/open_llama
A guidance language for controlling large language models. - guidance-ai/guidance
In the rapidly evolving field of AI, using large language models in an efficient and effective manner is becoming more and more important. In this article, y...
Created by researchers from UC Berkeley, CMU, Stanford, and UC San Diego, Vicuna is part of the new wave of models that use Meta's LLaMA as its foundation.
Your own LLM. MiniGPT-4. WebGPT on WebGPU. Transformers from scratch. ChatGTP Plugins demo live. Whisper JAX. LLaVA. MetaAI DINO SoTA Computer Vision. Autonomous agents in LangChain. RedPajama.
Sundays, The Sequence Scope brings a summary of the most important research papers, technology releases and VC funding deals in the artificial intelligence space.
An analysis of a chatbot data set by The Washington Post reveals the proprietary, personal, and often offensive websites that go into an AI’s training data.
A Cross-Section of the Most Relevant Literature To Get Up to Speed
Dolly 2.0 could spark a new wave of fully open source LLMs similar to ChatGPT.
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Dead simple way to run LLaMA on your computer
The article talks about R packages with a range of functions, including data manipulation, data visualization, machine learning, etc.
LLaMA-13B reportedly outperforms ChatGPT-like tech despite being 10x smaller.
An intuitive understanding of how AI-generated art is made by Stable Diffusion, Midjourney, or DALL-E
Explore data with Python & SQL, work together with your team, and share insights that lead to action — all in one place with Deepnote.
If you thought text-to-image AI was unbelievable, wait until you see how it compresses images.