A Helping Hand for LLMs (Retrieval Augmented Generation) - Computerphile

Comments · 146

• MO

  @mokopa · 2 years ago

  <a href="https://www.youtube.com/watch?v=of4UDMvi2Kw&amp;t=492">8:12</a> &quot;Langchain does a lot of other stuff that I&apos;m not using&quot;...langchain in a nutshell

  38

• TO

  @Tomyb15 · 2 years ago

  It&apos;s surprisingly bare bones as an approach. I was expecting something more sophisticated than just sticking the context as part of a promt and literally telling the model to use it in the answer. Reminds me of &quot;promp engineers&quot; sticking a <i>&quot;and please don&apos;t lie&quot;</i> at the end of a prompt to decrease hallucinations 😂

  50

• MI

  @mikoaj1321 · 2 years ago

  The presented example wasn&apos;t quite RAG. You&apos;re just putting more text into the context window. This method quickly falls short if you need to process a big set of reference data, like an entire PDF documentation. Real RAG is a bit more complicated and involves an additional step of converting the reference data to tokens that can be stored, then during inference you first convert the query to tokens, then find best matches with stored data, then use that search to generate excerpts from the original data to feed into your final inference window.

  140

• AL

  @alastairzotos · 2 years ago (edited)

  I worked on a RAG to make product recommendations, but eventually I was supplying it with too much data as context and it wouldn&apos;t work.<br><br>I settled on a neat solution: use GPT&apos;s ability to call functions and tell it something like, &quot;when the user asks for a recommendation, call the get_recommendations function with a summary of the user&apos;s query&quot;. It&apos;s cool that it gave me a summary because the embeddings are much better than those of a whole sentence or paragraph. So I could take that embedding and look up products based on semantic similarity to the user&apos;s query, while it was still generating a response, and then pass the top 10 back to GPT for it to show the user

  26

• KY

  @KylerChin · 2 years ago

  Feels illegal to be this early to Prof. Pound&apos;s lectures

  81

• MS

  @mscotty910 · 1 year ago

  Out of all the people they have Mike is the best (IMO) it would be awesome to do a segment with him on how models like Stable Video Diffusion Image-to-Video &nbsp;work

  3

• DU

  @dukestt543 · 2 years ago

  The word &quot;Strawberry&quot; actually has two R&apos;s. I apologize for any confusion caused earlier. - Chat GPT

  110

• AM

  @amrelmohamady · 2 years ago

  Now we need a video on fine tuning!

  27

• PE

  @penfold-55 · 2 years ago

  The problem with RAG and LLM&apos;s are the same. The risk is that the user takes what is said at face value.<br>Where RAG really can improve the situation is if the source is provided.<br>If you have a group of formal documents (such as documents for company procedure) then you should always state the source of that document.<br>This not only improves the trust of the model, but also narrows down where the user needs to look.<br><br>If it is just a black box, it can be hard for the user to know whether the RAG worked or whether it was hallucinating.

  63

• I1

  @i1abnrk · 2 years ago

  I remember having a whole box of the green printer paper. A family friend worked at the state and gave it to me for drawing, etc. some of it had phone numbers and addresses. Long ago in the city dump now.

  1

• GA

  @garcipat · 2 years ago

  Funny. just had to do this in a hackathon last week :)

• _D

  @_di_su · 1 year ago

  brilliant! thank you