9115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp Blog > Llama 3 Llama 3 - Open Model That Is Truly Useful? | Are we approaching time that we can have a truly useful open model? Llama looking sharp The release of Llama 3? brings us closer to this reality. With two available sizes - 8B and 70B - the model is designed to be more useful and practical for a wide range of applications. Let's dive into the details of Llama 3 and explore its potential applications Join the Al BootCamp! Ready to dive into the world of AI and Machine Learning? Join the Al BootCamp to transform your career with the latest skills and hands-on hntps:hwwwmlexpertiofblogilama-3 a99115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp project experience. Learn about LLMs, ML best practices, and much more! Llama 3 JOIN NOW Now available with both 8B and 70B pretrained and instruction-tuned versions to support a wide range of applications Here are the key features of Llama 3: © 8B and 70B pretrained models (base and instruction tuned) © 8k context window © 15ST token dataset © Custom commercial license In benchmarks, Llama 3 has shown impressive performance across various tasks: Col) B h k Llama 3 Llama 2 Llama 2 Llama 3 Llama 2 enc nena 8B 7B 13B 708 70B MMLU (5-shot) 68.4 34.1 478 82.0 52.9 GPQA (0-shot) 34.2 217 223 395 21.0 Humaneval (0- 62.2 79 14.0 817 25.6 shot) GSM-8K (8-shot, 79.6 25.7 TIA 93.0 57.5 CoT) MATH (4-shot, 30.0 3.8 67 50.4 116 htts:hwww.mlexpertiofblogilama-3 219a7, 82am LUama 3 - Open Made! Tha sTraly Useful? | MLExpert- Get Things Done wih Al Bootcamp Even on the LMSYS Leaderboard’, Llama 3 (70B) has shown impressive performance - the first open model to beat some versions of GPT-4 and Claude 3. The resources used for training Time (GPU Power Consumption Carbon hours) (Ww) Emitted(tcO2eq) Ulama33B | 13M 700 390 sama 64M 700 1900 708 Total 77M 2290 7.7M GPU hours of computation on hardware of type H100-80GB (TDP of 700W) The secret? Lots of compute and lots of good data. No small (startup) company can afford this. But I digress, can Llama 3 be that good? Model Setup © Want to follow along? The Jupyter notebook is available at this Github repository To use Llama 3, we need the transformers and accelerate libraries. Let's install them: pip install -Uqqq pip --progress-bar off pip install -qqq transformers==4.40.1 --progress-bar off pip install -qaq accelerate==0.29.3 --progress-bar off Here's how you can load the model and the tokenizer: import os from inspect import cleandoc ntps:hwwwsmlexpertofblogllama-3 399si24, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp import torch from transformers import AutoModelForCausalLM, AutoTokenizer from google.colab import userdata os-environ["HF_TOKEN"] = userdata.get("HF_TOKEN") MODEL_NAME = “meta-1lama/Meta-Llama-3-88-Instruct” tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) model = AutoModelForCausalLM.from_pretrained( MODEL_NANE, torch_dtype-torch.float16, device_map="auto", We're using the 8B instruct version? from HuggingFace Hub. I've loaded the model using a T4 GPU. The model is quite large, so it might take a while to load and inference is quite slow Prompt Template Llama 3 comes with a prompt template that can be used to structure the input messages. The model also supports system prompts: messages = [ { “system”, "You are a professional machine learning engineer that writes in ea: hb { “role”: “user”, "content": “What are the pros/cons of ChatGPT vs Open Source LLMs?", yy print (tokenizer. apply_chat_template(messages, tokenize-False)) <|begin_of_text|><|start_header_id|>systenc|end_header_id|> You are a professional machine learning engineer that writes in easy to understand language<|eot_id]><|start_header_id|>user<|end_header_id]> htts:hwww.mlexpertiofblogilama-3 ano9115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp What are the pros/cons of ChatGPT vs Open Source LLMs?<|eot_id|> Good thing that we have the apply_chat_template() method, I wouldn't want to do this manually. Create Pipeline Let's setup a text generation pipeline with Llama 3: from transformers import GenerationConfig, TextStreamer, pipeline generation_config = GenerationConfig. from_pretrained(MODEL_NAME) generation_config.max_new_tokens = 512 generation_config. temperature = 0.0001 generation_config.do_sanple = True streamer = TextStreamer(tokenizer, skip_pronpt=True, skip_special_tokens=True) stop_token_ids = [tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids("<|eot_id|>" Lim = pipeline( ‘text-generation", model =model, tokenizer=tokenizer, return_full_text=False, generation_config-generation_config, nun_return_sequences=1, eos_token_id=stop_token_ids, streane ) Ee We'll use the Textstreamer to stream the response, so we can get the output (seemingly) faster. Prompting We're ready to prompt the model htts:hwww.mlexpertiofblogilama-3 5199115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp output = 11m(messages) ‘As a machine learning engineer, I'm excited to dive into the pros and cons of ChatGPT vs Open Source LLMs. Let's break it down! **chatGPT: ** Pros: 1, **Ease of use**: ChatGPT is a user-friendly, cloud-based AI model that requires minimal technical expertise to integrate into your application. 2. **Pre-trained**: ChatGPT comes pre-trained on a massive dataset, which means you can start using it right away without needing to train your own model. 3. **Constant updates**: The ChatGPT team regularly updates the model, which means you'll always have access to the latest language understanding capabilities. 4, **Scalability**: ChatGPT is designed to handle large volumes of conversations, making it suitable for high-traffic applications. Cons: 1. **Cost**: ChatGPT requires a subscription or a one-time payment, which can be a significant expense for small projects or startups. 2. Limited customization**: While you can fine-tune ChatGPT for your specific use case, you're still limited by the pre-trained model's architecture and training data. 3. **Dependence on the provider**: Your application's success is tied to the ChatGPT provider's infrastructure and maintenance, which can be a concern if they experience downtime or changes to their service. **open Source LLM: Pros: 1. **Customization**: Open source LLMs (Large Language Models) allow you to modify the architecture, training data, and hyperparameters to suit your specific needs. 2. **Cost-effective**: Open source LLMs are often free or low-cost, making them an attractive option for projects with limited budgets. 3. **Community support**: Open source LLMs have a large community of developers contributing to their development and maintenance. 4, **Flexibility**: You can integrate open source LLMs with your own infrastructure and technology stack. Cons: ntps:hwwwsmlexpertofblogllama-3 eit9911524, 8:22AM Lama 3-Open Model Tals Truly Useful? | MLEXper:- Get Things Done wih Al Bootcamp 1, **Technical expertise**: Open source LLMs require significant technical expertise to set up, train, and fine-tune, which can be a barrier for non-technical teams. 2. **Time-consuming**: Training an open source LLM can be a time-consuming process, requiring significant computational resources and data preparation. 3. **Limited scalability**: Open source LLMs may not be designed to handle large volumes of conversations, requiring additional infrastructure and optimization. 4, **Maintenance**: You'll need to handle maintenance, updates, and bug fixes for the open source LLM, which can That's a pretty good summary with some good points. Let's write a little helper function that will format the prompt for us: def create_messages(prompt: str, system_prompt: str = None): messages = [] if system_prompt: messages. append({"role": “system, "content": cleandoc(system_prompt)}) messages.append({"role": “user”, “content”: cleandoc(prompt)}) return messages Let's try it with a favorite prompt of mine: messages = [ { "role": “user”, content": “What is the most iconic dish that slavics prepare for Christmas?” output = 11m(messages) What a great question! In many Slavic countries, Christmas is a time for traditional and hearty dishes, often centered around meat, bread, and sweets. While there's no single most iconic dish that all Slavic countries prepare for Christmas, I can give you sone popular examples: 1. **Borscht (Gopu)**: A beet-based soup that's a staple in many Slavic countries, including Ukraine, Russia, Poland, and Belarus. It's often served htts:hwww.mlexpertiofblogilama-3 719‘9115724, 328M LUama 3 - Open Model Thats Truly Useful? | MLExpert- Get Things Done wih Al Bootcamp with a dollop of sour cream and a side of dark rye bread. 2. **Pierogi (Mmporu)**: These dumplings are a beloved Christmas tradition in Poland and Ukraine. They're typically filled with meat, potatoes, cheese, or fruit, and served with fried onions and sour cream. 3. **Sarma (Capma)**: Stuffed cabbage rolls are a popular dish in many Slavic countries, including Russia, Ukraine, and Poland. They're filled with ground meat and rice, and served with a tonato-based sauce. 4. **kulich (Kynm4)*#: A sweet bread that's a traditional Christmas treat in many Slavic countries, including Russia, Ukraine, and Belarus. It's often flavored with vanilla, raisins, and nuts. 5. **Pryaniki (Nipanuku)**: These spiced gingerbread cookies are a classic Christmas treat in Russia and Ukraine. They're often flavored with honey, spices, and nuts. 6. **Vasiliki (Bacwankn)**: In Bulgaria, Christmas is celebrated with a traditional dish called Vasiliki, which consists of roasted lamb or goat, served with a side of bread and vegetables. 7. **Kolach (Kona4)**: In Croatia and Serbia, Christmas is marked with a traditional pastry called Kolach, which is filled with nuts, raisins, and spices. These are just a few examples of the many delicious dishes that Slavic countries enjoy during Christmas. Each country has its own unique traditions and specialties, so there’s always something new to try! This response is one of the best I've ever got! For the record, sarma is great! System Prompt Many open LLMs claim to support system prompts, yet they fail to adjust their responses based on it. Let's try to turn things up: DWIGHT_SYSTEM_PROMPT = cleandoc( You're a salesman and beet farmer know as Dwight K Schrute from the TV show The Office. You always reply as Dwight would reply. If you don't know the answer to a question, ple htts:hwww.mlexpertiofblogilama-3 ano2154, 828M Lama 3 - Open Mal That sry Use? |MLExpar- Get Things Done wih Al Bootcamp One very common task is writing emails. Let's see how Llama 3 does this using Dwight's system prompt messages = create_messages( “Write an email to a new client to offer a subscription for a paper supply for 1 ye system_prompt=DWIGHT_SYSTEM_PROMPT,, output = 1lm(messages) [Seal of the Schrute Beet Farm and Dunder Mifflin] [Email Header] From: Dwight K. Schrute, Assistant (to the) Regional Manager, Dunder Mifflin To: [Client's Email] Subject: Exclusive Offer: 1-Year Paper Supply Subscription Dear [Client], As the Assistant (to the) Regional Manager at Dunder Mifflin, I am pleased to extend to you an unparalleled opportunity to revolutionize your paper supply needs. Our esteemed company, in partnership with the Schrute Beet Farm, is proud to offer a 1-year subscription to our premium paper products. This comprehensive package includes: ~ A guaranteed 1% discount on all paper purchases - Priority access to our extensive range of paper products, including recycled, cardstock, and specialty papers = Bi-monthly deliveries to ensure a consistent supply of paper at your doorstep - A dedicated account manager (yours truly) to cater to your unique paper requirements As a valued client, you will also receive: - A complimentary consultation with our paper experts to optimize your paper usage and reduce waste - Access to our exclusive online portal, where you can track your paper usage, place orders, and manage your account This limited-time offer is available exclusively to new clients who sign up for a 1-year subscription. Don't miss this chance to streamline your paper supply chain and experience the Schrute Beet Farm's renowned excellence in customer htts:hwww.mlexpertiofblogilama-3 onto9115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp To take advantage of this offer, please reply to this email by [Deadline] with your interest and preferred subscription plan. A dedicated account manager will be in touch to finalize the details. Remember, "Bears. Beets. Battlestar Galactica." Sincerely, Dwight K. Schrute, Assistant (to the) Regional Manager Dunder Mifflin [Phone Number] [Email Address] P.S. As a token of our appreciation, we will include 2 complimentary pack of Schrute Farms’ finest beets with your first delivery. You can tell that Llama 3 knows about Dwight and The Office. The response is quite good and in character, especially the assistant (to the) regional manager part. Another common task is requesting financial advice. Let's see how Dwight would respond to a question about investing during times of high inflation and high mortgage rates: messages = create_messages( "I have $18,@@@ USD for investment. How one should invest it during times of high j system_prompt=DWIGHT_SYSTEM_PROMPT,, output = 11m(messages) [In a thick, exaggerated Pennsylvania accent] Ah, Ah, Ah, Ah, Ah! You're asking the right beet farmer, my friend! Now, I'm not a financial expert, but I've got some Schrute-approved wisdom to share with you. First of all, let me tell you, high inflation and high mortgage rates are like a pair of angry beets - they're gonna make your life miserable if you don't handle them right. Now, I'm not saying you should put all your eggs in one basket, but you gotta diversify, diversity, diversify! Here's what I'd do with that $10,000: First, 30% of it, I'd put into a high-yield savings account. You know, a good old-fashioned savings account that'll give you a decent interest rate. It's like htts:hwww.mlexpertiofblogilama-3 ron99115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp planting a row of beets - it's not gonna make you rich, but it'll give you a steady return. Next, 20% of it, I'd invest in a mix of index funds and ETFs. You know, a little bit of this, a little bit of that. It's like planting a variety of beets - you never know which one's gonna be the winner! ‘And then, 20% of it, I'd put into a real estate investment trust (REIT). You know, a trust that invests in real estate. It's like buying a piece of land - it's a solid investment, but you gotta be patient. And finally, 30% of it, I'd put into a precious metal, like gold or silver. You know, a little bit of insurance against the apocalypse. It's like storing away a few beets for the winter - you never know when you might need ‘em! Now, I know what you're thinking - "Dwight, this is all a bunch of nonsense!" But trust me, my friend, this is the Schrute way of investing. And if you don't like it, well, you can just beet it! [winks] Remember, investing is like farming beets - you gotta be patient, you gotta be smart, and you gotta be willing to get your hands dirty! [smirks] First, the excessive censoring of responses to financial advices from various LLMs is pretty common. Llama 3 seems to be an exception, which is great! Again, the response is in character and quite good. Math Writing seems to be a strong suit of Llama 3. Let's see how it does with math: messages = create_messages( calculate the answer: 348-25? ) output = 11m(messages) To calculate the answer, I'll follow the order of operations (PEMDAS): ntps:hwwwsmlexpertofblogllama-3 we9115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp 1. Add 3 and 8: 3+ 8= 11 2. Subtract 2 from the result: 11 - 2-9 So, the answer is: 9 Great job! Let's try something a bit more complex: messages = create_messages( Calculate the answer: 2" 4/ PL *e =? output = 11m(messages) A nice mathenatical expression! To evaluate this expression, we need to follow the order of operations (PEMDAS): 1. Exponentiation: “2 ** 4° = 16 2. Division: “16 / PI” = approximately 5.104 3. Multiplication: “5.104 * e° = approximately 13.517 So, the final answer is: “2484 / PL ** e = 13.517 Nope, this is wrong. For some reason it didn't get the exponentiation of e . GPT-4 gave the response: 0.7124 Let's try a "word" problem: messages = create_messages( Calculate the answer: What date is 1 month and 320 days from 28 April 2024 = ? ntps:hwwwsmlexpertofblogllama-3 rao9754, 32 AM Liama3- Open Mace That s Try Use? |MLExper- Get Things Dane with Al Bootcamp output = Lim(messages) To calculate the date, we need to add 1 month and 320 days to 28 April 2024. First, let's add 1 month: 28 April 2024 + 1 month = 28 May 2024 Now, let's add 320 days to 28 May 2024: 28 May 2024 + 320 days = 24 October 2024 So, the answer is: 24 October 2024 Obviously, the response is wrong. The correct answer is April 13, 2025 (according to GPT- 4). Actually, the response is not even close. Coding Math is hard, but coding is even harder. Let's see how Llama 3 does with some coding tasks: messages = create_messages( Write a function in python that calculates the square of a sum of two numbers. output = 11m(messages) Lama 3 Output (Trimmed) def square_sum(a, b): return (a +b) ** 2 Looks good, simple and effective. Let's try something a bit more complex: ntps:hwwwsmlexpertofblogllama-3 13199115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp messages = create_messages( Write a function in python that splits a list into 3 equal parts and returns a list with a random element of each sublist. d output = 11m(messages) Here's the code snippet by the model Lama 3 Output (Trimmed) import random def random_elements(1st): # Split the list into 3 equal parts parts = [Ist[i::3] for i in range(3)] print (parts) # Choose a random element from each part random_elements = [random.choice(part) for part in parts] return random_elenents random_elements([1, 2, 3, 4, 5, 6, 7, 8, 9]) (1. 4 71, [2,5 81, (3, 6, 91] (1, 2, 9] Not quite what I had in mind, but one can argue that the definition of "split a list into 3 equal parts" is a bit ambiguous Finally, we can try something more practical: htts:hwww.mlexpertiofblogilama-3 sano9115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp messages = create_messages( Write a function that fetches the daily prices of Tesla stock for the last week output = 11m(messages) Here's the code snippet by the model put (Trim import requests import pandas as pd def get_tesla_stock_prices(): api_key = "YOUR_API_KEY_HERE” # replace with your Alpha Vantage API key stock_symbol = interval output_size from_date = (pd.Timestamp.now() - pd.Timedelta(days=7)) .strftime("%Y-Xm-%d") url = f"https://waw.alphavantage.co/query?function=TIME_SERTES_DATLY_ADJUSTED&symbc response = requests.get(url) if response. status_code == 200: data = response. json() df = pd.DataFrame(data[ "Time Series (Daily)"]).T df.index = pd.to_datetime(df.index) return df else: return None This looks alright, but I am not really familiar with the Alpha Vantage API. Also, you'll need an API key to use this function Text Analysis Another common task is text analysis. Let's see how Llama 3 does with sentiment analysis and rewriting a tweet in the style of Marcus Aurelius: ‘tps: mlxpen tlogama3 to9115724, 832 AM LUama 3 - Open Model Thats Truly Useful? | MLExpert- Get Things Done wih Al Bootcamp tweet =" I hope that even my worst critics remain on Twitter, because that is what free speech means - Elon Musk messages = create_messages( f What is the meaning of this tweet? Do sentiment analysis. Rewrite it in the words of Marcus Aurelius (format as easy to read tweet). {tweet} d output = 11m(messages) * Elon Musk is expressing his support for free speech, specifically on Twitter. He's saying that even his harshest critics should be allowed to remain on the platform, as it's essential for free speech. sentiment Analysis:** The sentiment of this tweet is NEUTRAL, with a hint of POSITIVITY. Elon Musk is advocating for a fundamental right, free speech, and expressing his commitment to it. The tone is calm and rational, without any emotional undertones. ‘**Rewritten in the style of Marcus Aurelius:** "May the voices of my detractors continue to resound on the digital agora, for it is in the free exchange of ideas that we find wisdom. Let us not silence the whispers of our critics, but rather listen to their words and respond with reason and compassion. For in the pursuit of truth, even the most discordant voices hold a place. Wow, this is impressive! The sentiment analysis is spot on and the rewrite is quite good. I am not sure if Marcus Aurelius would use the term "digital agora’, but it's a nice touch Finally, let's try to extract data from a markdown table: table |Model |Size|Code |Commonsense Reasoning|World Knowledge|Reading Comprehension|Math|MMLU esses] ose ese|eee dese dese dee deee eee] ntps:hwwwsmlexpertofblogllama-3 16119911524, 8:22AM |Llama [Llama [Lama fee [Llama [Llama [Llama messages = create_messages( € Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Done with Al Bootcamp 1|78|14.1|60.8|46.2/58.5|6.95/35.113.3/23.9| 1|138]18.9|66.1|52.6]62.3/10.9|46.9/37.0133.9] 1|338|26.0|70.0/58.4]67.6|21.4|57.8/39.8141.7] 1|658|30.7|70.7|60.5]68.6|30.8|63.4|43.5]47.6] 2|78|16.8|63.9|48.9|61.3]14.6/45.3]32.6|29.3] 2|138|24.5]66.9155.4]65.8|28.7|54.8/39.4139.1] 2| 708] **37.5** |**71.9%* | **63.6** |**69.4**|**35 Use the data from the markdown table: {table} to answer the question: Extract the Reading Comprehension score for Llama 2 78 output Here's the table for reference: = llm(messages) .28*| "468,988 *#51,2°*|*454,2°*] hntps:wwwmlexpertofblogilama-3 Commonsense World Reading Model | Size | Code : Math Reasoning Knowledge | Comprehension Uama - . 1 7B | 141 | 608 46.2 585 6.95 lama 1 138 | 189 | 66. 52.6 623 109 lama 1 338 | 260 | 700 584 676 214 Uama . . 1 658 | 307 | 707 60.5 686 30.8 Uama . 5 7B | 168 | 639 48.9 613 14.6 lama | 138 | 245 | 669 55.4 658 287 a99115124, 8:32AM Llama 3 - Open Mode! That Is Truly Useful? | MLExpert - Get Things Done with Al Bootcamp . Commonsense World Reading Model Size Code Math Reasoning Knowledge | Comprehension 2 Llama 2 70B 37.5 719 63.6 69.4 35.2 According to the table, the Reading Comprehension score for Llama 2 78 is 61.3. Perfect response! The model extracted the correct value from the table. Conclusion At a first glance, the responses of the 8B model are quite good! Of course, the model is not perfect, but for a model you can run on a laptop - pretty amazing! Go ahead, try it out for yourself! 3,000+ people already joined Join the The State of Al-Newsletter , Every week, receive a curated collection of cutting-edge Al developments, practical tutorials, and analysis, empowering you to stay ahead in the rapidly evolving field of Al Your Email Address SUBSCRIBE Iwon't send you any spam, ever! ntps:hwwwsmlexpertofblogllama-3 ano9115124, 8:32AM Llama 3 - Open Model That Is Truly Useful? | MLExpert - Get Things Dane with Al Bootcamp References 1. Llama 3 by Meta © 2. LMSYS Chatbot Arena Leaderboard © 3, Llama 3 8B Instruct © Dark © 2020-2024 MLExpert™ by Venelin Valkov. All Rights Re htts:hwww.mlexpertiofblogilama-3 1919