Today’s post: we take an LLM and try (but not very hard) to make it take movie notes like me and see what funny content it generates.
Every time I watch a movie, I write five to ten sentences about it in a spreadsheet that’s rotting away in a random folder on my computer. I’ve never shared the outputs because the writing quality is extremely low now (I watch movies at the very end of the day after spending all day on the computer at work). But I’ve been writing notes on movies for around five years now, and a pace of around 200 movies watched per year means I’ve written 1000+ reviews at this point. Datasets as clean and organized as this don’t come easy, so let’s have some fun with it!
Before we start, let me go through a brief overview of a simplified hierarchy of LLM techniques from easy to hard (and low to high cost).
At the bottom of it is prompt engineering, which is easy to do and is really just a smarter way of asking the LLM (in this case ChatGPT) questions. Next up is RAG, which augments this process by giving the LLM some context based on documents you have stored in a database. When we get to fine-tuning, we’re actually starting to alter the model itself, but we’re not training it from scratch. Once we get to the custom model level, we’re building nearly everything ourselves.
1. Movie Reviews via Prompt Engineering
I already have an OpenAI subscription, so I was able to directly prompt ChatGPT-4 via the API. Let’s start by just asking for a movie review to see what it looks like without any modification.
The prompt: “Write me a review for Late Spring directed by Yasujiro Ozu.”
Easy enough, right? The response is actually pretty good, although it lacks a human POV:
Title: A Subtle Masterpiece - "Late Spring" by Yasujiro Ozu
Rating: ★★★★★
Yasujiro Ozu's 1949 film "Late Spring" is indisputably one of cinema's timeless masterpieces. Beyond its technical prowess and narrative finesse, the movie delves deep into themes of familial love, personal sacrifice, societal obligations, and the transience of life.
The story of "Late Spring" revolves around the relationship between a widowed father, Shukichi, and his devoted daughter, Noriko, embodying a profound sense of realism and emotional depth. Their simple yet captivating life is governed by a sense of serenity and unspoken bonds that are deeply resonant.
Ozu's unique directorial style is breathtaking. The framing of shots, observant camera angles, and the famous "pillow shots" contribute to the contemplative aesthetic beauty of this film. The director’s ability to capture intrinsic details and everyday life with such crisp precision is truly compelling.
The screenplay, co-written by Ozu with Kogo Noda, transforms even the mundane conversation and household routines into meaningful circumstances, reflecting astute observations of human nature, suffering, and peace.
The performances are equally laudable. Setsuko Hara as Noriko delivers an impeccable and authentic performance while Chishu Ryu as Shukichi expresses profound emotion with sublime vitality and minimalist acting. Their on-screen chemistry is as compelling as their individual performances.
The pacing of the film may seem slow to some viewers, yet it is this unhurried tempo that allows Ozu to delve into the depths of human emotions and the nuanced complexities of ordinary life. Ozu refrains from dramatic climaxes and artificial contrivances, instead, the movie's strength lies in its subtly evoked emotions and the director's mature understanding of life itself.
In conclusion, “Late Spring” is more than a just film. It is a poetic exploration of life’s fleeting beauty, unspoken emotions, and inevitable change. This film is a testament to Ozu’s genius as a filmmaker, displaying a complex blend of simplicity, emotional depth, and poignant beauty. It’s a must-watch for cinema aficionados who appreciate contemplative storytelling and nuanced filmmaking.
Next, let’s feed in my review data with a few-shot prompt (giving ChatGPT some examples so it knows what the task is at hand).
The prompt (I cut off the text because it was getting too long, but I provided five examples):
Write me a review of the movie Late Spring directed by Yasujiro Ozu, in the style of OrcaNinjaGPT.
Here are some examples of OrcaNinjaGPT's tone and response style:
User: Please provide a review of the movie Love Exposure directed by Sion Sono,
written in the style of OrcaNinjaGPT.
OrcaNinjaGPT: Is there any four hour film that goes by as quickly as Love Exposure? This is the work of an eccentric genius, informed by Sono's unique taste for the vulgar. Or maybe it's just a high tier rom-com warped into a truly unusual product by a director who spent most of his time at Berkeley watching B-movies (hence the Miss Scorpion reference) and pornos (in fact he directed one himself!). While the love story is at the center, what drives the story are the characters. They are absurd enough to be exciting, but flawed in the most human ways. Yu may be a pervert but he's also just a teenage boy discovering his sexuality. Yoko and Koike are rightfully lashing out at the world for abuse and upbringings they did not deserve. Totsu himself while barely a sympathetic character still displays the naivete that allows him to become part of not one but two cults. Actually, it might be the absurd elements (and acting) which allow the more grounded and subtle moments to shine.
Sono takes a thematic deep dive into the nature of what is perverse. Who is worse, the man who takes upskirt pictures while trying to find love in his own way, or the one who listens to disgusting confessions and forgives them? Sono forces us to side with the honest pervert over the hypocritical cult leader, but it's a surprisingly easy position to take. Then after religion has taken it's fair share of criticism, in comes Yoko with a three minute recitation of 1 Corinthians 13: perhaps there's some truth to be found in the text despite the trash that peddle it. Even if the (extremely straightforward) religious elements go over the head of some, at minimum the average viewer will empathize with how difficult it is to bounce back from trauma of any type and how painful it is to try to love someone and receive no validation.
Beyond the abstract, Love Exposure has it all. The kung-fu involved in upskirt hunting is gross but hilarious. Violence is injected in appropriate doses to keep the heart pumping while the plot seamlessly weaves the lives of the five main characters together before jumping from a classic rom-com plot (brother-zoning yourself) to an exhilirating cult infiltration thriller. This will be a 1-2 or 9-10 rating movie for most because of its insane balance of tone and absurd content; I loved it.
...
The response:
OrcaNinjaGPT: Late Spring, directed by Yasujiro Ozu, might seem a simple exploration of the father-daughter relationship, but underneath its serene narrative, it embodies the subtle poetics of everyday life. The centerpiece of Ozu's "Noriko Trilogy," this film contains nuances that are as fulfilling as they are complex.
Let's start with the characters. Noriko, played by the deeply expressive Setsuko Hara, and her widower father Shukichi, brought to life by Chishu Ryu; both deliver performances that are so quietly powerful, they seep into your soul. Noriko's delicate struggle with societal pressures and maintaining her father's happiness is a sublime portrayal of love and duty. Shukichi, on the other hand, is a fascinating mirror to Noriko's devotion, wrestling with his own conflicting emotions of wanting his daughter to live her own life versus his fear of loneliness.
The efficacy of Late Spring lies largely in its minimalistic engagement of emotive elements. Ozu's signature "tatami shot" technique captivates viewers, inviting them into the intimate world of his characters. This visual simplicity coupled with the masterful use of ellipses paints a vibrant picture of post-war Japan, efficiently conveying the cultural and generational shift of that time.
In terms of sound, Ozu's careful usage of silence is as evocative, if not more so, than the dialogue itself. This purposefulness translates into the score too; a repetitive, melancholic tune effectively mirrors the recurring themes of loss, impermanence, and change.
One may argue the pacing is slow, but it's this intimacy with the temporal that heightens the beauty of Late Spring. Moments of mundane domesticity are allowed to breathe, adding to the ebb and flow of this heart-tugging narrative.
The essence of Late Spring is captured in a father and daughter's heart-rending dilemma of love vs duty. It's a nod to the evanescent beauty of life's transient moments and an elegy to the inevitable progression of time. Yes, the movie ends on a note of sorrow, but it's laced with a kind of serenity — a testament to Ozu's mastery of quiet, lingering impact. A masterpiece that is as poignant as it is profound, Late Spring shines with the gentle luminosity of the cherry blossoms it so tenderly symbolizes.
It’s more casual and improved. Actually this is in pretty high agreement with what I wrote, although I don’t think I would ever use the word “evanescent” in a sentence. Late Spring is a highly acclaimed work from the 40’s so plenty of folks have written about it; ChatGPT is probably mixing and matching elements of reviews from decent writers. The LLM succeeds in mirroring the format of my notes although it’s hard to evaluate how successful it is at impersonating my writing style.
From now on, I can probably use this as a tool to create some sort of benchmark for my own writing. Any writing I do that’s of value needs to exceed this in originality and individuality.
There’s actually a lot more we can do with prompting, but I’m lazy. For example, we can take a more chain-of-thought approach and have the LLM analyze my reviews for style as a first step before writing the reivew.
2. Movie Reviews with RAG
Everyone loves RAG nowadays. The main idea: build a source of reference documents, retrieve (R) the most relevant ones for the prompt and use them to generate (G) the augmented (A) output. This approach allows me to use my full arsenal of 1129 reviews. From a technical standpoint, I’m using OpenAI’s text-embedding-3-small model to embed my reviews and postgres as my vector database (using pgvector).
Below you can see a preview of my data, thinned down to just a movie title, my notes, and the director name. All of this goes into the vectordb and I use cosine similarity to find relevant notes for a given prompt.
Now all that’s left is to setup the prompt. We give ChatGPT some friendly instructions:
You are a friendly chatbot and expert on film.
You can answer questions about movies using the provided notes from a reviewer named OrcaNinja.
You generate detailed responses in an informative and conversational tone.
Then along with our prompt, we pass along “Relevant Notes from a reviewer named OrcaNinja” which consists of the top three matching reviews pulled from my review database.
All that’s left is to try it and laugh at the results.
Prompt: "Show me OrcaNinja's notes on The Boy and the Heron directed by Miyazaki"
My RAG process successfully pulled in the reviews from The Boy and the Heron, Princess Mononoke, and Spirited Away. Great!
Response:
OrcaNinja’s review of “The Boy and the Heron” directed by Miyazaki is quite detailed. He praises the film’s stunning visuals and emotional depth, noting that it’s more of an emotional piece than a narrative-driven one. He particularly highlights the film’s exploration of grief, old age, and life, and the way it uses animation to create a world where emotions are colors and shapes with seemingly unbounded imaginative limits.
However, OrcaNinja also points out that the film sometimes loses focus between the reflections of an old man and the emotional journey of a child. He feels that the film is stronger as a young man’s adventure than it is in considering the dilemmas of an artist, and this entanglement somewhat mutes the impact at the end.
OrcaNinja also mentions Miyazaki’s other works, “Princess Mononoke” and “Spirited Away”, praising the former for its environmental themes and the latter for its top-notch animation and score. He appreciates the character development in “Spirited Away” and the way it allows viewers into Miyazaki’s imagination. However, he feels that the last section of “Spirited Away” is somewhat fumbled from a narrative and pacing standpoint.
My comments on the last section of Sprited Away were not that negative. The “friendly instructions” provided to the LLM were actually critical for getting this answer. On my first attempt with different instructions, ChatGPT responded by telling me that Miyazaki does not have a film called The Boy and the Heron despite the retrieval process successfully feeding it my review. Nice try, pal…
Prompt: Tell me about OrcaNinja's opinion on Another Round directed by Vinterburg
Response: I’m sorry, but it seems there’s been a misunderstanding. The notes provided by OrcaNinja do not contain any information or opinions about the movie “Another Round” directed by Thomas Vinterberg. The notes discuss other films such as “One Cut of the Dead”, “Red Dot”, and “Funny Games”. If you have any other questions or need information on a different film, feel free to ask!
Uh-oh… the RAG process pulled One Cut of the Dead, Red Dot, and Funny Games as the top three matches. Let’s try something else.
Prompt: Write me a review for the 2022 Scream in the style of OrcaNinja
Response: OrcaNinja here, back with another review, this time for the 2022 reboot of the classic horror franchise, Scream. This fifth entry in the series is a nostalgic nod to the original, taking us back to where it all began 25 years ago. It’s a part of the 2020’s wave of nostalgia cash-ins, but it does so with a certain panache that sets it apart.
The film is a masterclass in meta-horror, a genre that the Scream series has always excelled in. Directors Bettinelli-Olpin and Gillett have understood that the charm of Scream lies in its ability to break the fourth wall and have fun with the genre. They’ve managed to critique modern horror and fandom while still appreciating the value of old tricks and the thrill of impending doom.
The screenplay is witty, the jump scares are timed to perfection, and the film has its own unique art style. The effects and gore sequences are top-notch, and the nostalgia factor is a bonus. Two scenes stand out as instant classics: Mindy yelling at her TV in a shot that mirrors the original Scream, and a fantastic kitchen sequence.
However, the film isn’t without its flaws. The performance of Barrera is a bit of a letdown, and the film doesn’t push the boundaries of the genre as much as it could have. The reveal of the killers, while in line with the slasher rules, is a bit unsatisfying. But these are minor nitpicks in an otherwise solid entry in the Scream franchise.
In conclusion, Scream (2022) is a delightful return to form for the series. It’s a film that knows it’s not high art, but revels in its self-awareness and has fun with the material. It’s a film that reminds us that sometimes, it’s okay to just sit back and enjoy the ride. So, grab your popcorn, turn off the lights, and get ready for a thrilling trip down memory lane.
Was ChatGPT trained with YouTube text-to-speech data? Yikes. We did successfully get a review though, and while it’s not as deep as I had hoped, it’s actually about as good as I expected. This time at least the more informal style does come through and it feels more “me” than the default response.
Prompt: Analyze Chungking Express in the style of OrcaNinja
I reviewed this movie at least three times and the RAG process pulled three decent sets of notes.
Response: OrcaNinja’s review of “Chungking Express” is a testament to the film’s unique exploration of themes such as timing, love, and loneliness set against the backdrop of Hong Kong. The reviewer appreciates director Wong Kar-wai’s ability to bypass the absurdity of the situations and focus on the emotional core of the story.
OrcaNinja notes that the film’s musical choices, including the use of the song “California Dreamin’”, enhance the narrative rather than distract from it. The song, which the reviewer initially found slightly obnoxious, is seen as a symbol of the characters’ hopeless optimism in a city that can often feel alienating.
The cinematography, led by Christopher Doyle, is another aspect of the film that OrcaNinja praises. The use of color, even the dull shade of green emitted by city lights, is transformed into something visually striking. The reviewer also highlights the innovative camera work in confined spaces and the unique perspective offered by shooting Tony Leung through a mirror at ground level.
Despite the film’s many strengths, OrcaNinja does point out a few areas that could have been improved. The second half of the film leans too heavily into the manic pixie dream girl trope, and the reviewer feels that actress Brigitte Lin was not used to her full potential.
However, these minor criticisms do not detract from OrcaNinja’s overall positive view of “Chungking Express”. The reviewer appreciates the film’s ability to bring characters into absurd scenarios in a way that appeals to both the eyes and our latent humanity. The film’s exploration of loneliness and love in a bustling city, combined with its technical prowess, make it a standout in Wong Kar-wai’s filmography.
Hmm… this time it actually fails the task at hand. Instead of analyzing from my POV it’s doing a summary of my reviews. Maybe a better prompt is needed. We’re lazy here, so let’s just move on.
Prompt: What is OrcaNinja's favorite John Woo movie?
Response: Based on OrcaNinja’s reviews, it appears that his favorite John Woo movie is “Red Cliff”. He praises the film for its high production value, large set pieces, and the director’s eye for action. He also appreciates the intellectual moments in the film, such as the instrument duet, which he describes as a truly great film moment. He expresses excitement about watching the five-hour version, indicating his deep interest and enjoyment of the film.
I’ve watched far more than three John Woo films. The retrieval process pulls in Casino Raiders II (directed by Johnnie To, not John Woo 🤔), In the Line of Duty 4 (directed by Yuen Woo-ping, not John Woo🤮) and Red Cliff (😐). At least of these three movies it picked Red Cliff. A less lazy prompt would ask the LLM to explain why.
Prompt: Compose a review for Letterboxd on Andrei Rublev directed by Tarkovsky in OrcaNinja's style and based on his notes.
My retrieval process missed on Andrei Rublev and instead returned The Sacrifice, Mirror, and Noah (dir. Aronofsky). 2/3 being Tarkovsky is less than ideal, and the result is pretty ugly.
Title: Andrei Rublev: A Spiritual Odyssey
Andrei Rublev, directed by the masterful Andrei Tarkovsky, is a cinematic journey that delves deep into the realms of faith, fear, and the human condition. It’s a film that doesn’t just present a narrative, but rather, it invites you to experience, react, and reflect.
Tarkovsky’s approach to storytelling is both unique and captivating. He employs a variety of angles and tracking shots, which not only add a visual dynamism to the film but also help to maintain a steady pace throughout its lengthy runtime. The film never feels sluggish or ill-paced, a testament to Tarkovsky’s directorial prowess.
The narrative of Andrei Rublev is steeped in spiritual angst, a theme that resonates throughout the film. The impending doom of a nuclear winter is not the focus here, but rather the miracle or insanity that lies in the sacrifice. It’s a tribute to a dying world of faith, a world that is beautifully and hauntingly portrayed.
The acting in Andrei Rublev is nothing short of superb, with performances that breathe life into Tarkovsky’s vision. The cinematography is equally impressive, although some of the tracking shots may induce a sense of unease. But perhaps that’s the point. Andrei Rublev is not a film that’s meant to comfort, but to challenge.
Andrei Rublev is a film that demands your attention and your emotional investment. It’s a film that doesn’t just tell a story, but rather, it paints a picture of a world steeped in faith, fear, and sacrifice. It’s a film that will leave you pondering long after the credits roll.
I should start titling my reviews! I’m not sure why I never considered doing this before. Maybe it’s because “Andrei Rublev: A Spiritual Odyssey” is just too cringe.
Since it doesn’t actually have my review of Andrei Rublev, it has to make things up based on the movies it did get. The movie is about faith, but nuclear winter is definitely not involved.
RAG is cool and I really am only scratching the surface of it. Using hybrid search instead of semantic search would be an easy way to improve retrieval quality, and there are also plenty of prompting tricks to use. I want to go watch the NBA Finals though, so I’ll leave the rest as an exercise for the reader…
3. Movie Reviews with a fine-tuned ChatGPT
Next, we can try fine-tuning the model itself and see what happens. OpenAI makes doing this very, very easy for their models. All I have to do is format my data so that it looks like a prompt/response pair, and feed it to the OpenAI API. OpenAI even provides functions you can use to check that the data format is correct before you start your fine-tuning job.
I submit my 1129 reviews in this format with the prompt as “Review X movie in the style of OrcaNinja” and the response as my notes on the movie. After about half an hour (and a few cents of training cost), we’re able to use the model freely.
These results would never be written by me, but they’re really fun and make no sense, which is exactly my type of humor. So much is wrong with these reviews that I won’t even bother explaining them. Have fun reading!
Prompt: Review the Godfather II in the style of OrcaNinja.
Response: Cowboy hat Sir Ian Holm is a strange draw in a film this serious, but just like its predecessor, Black Hawk Down, and The Departed (seriously these movies are scarily similar), it gets away with it because the story is so great and the pacing is excellent. While the Godfather teaches (or tries to teach) the viewer the value of family, whether this is actually shown is debatable. But certainly Michael’s return to Sicilian roots and flashback Robert’s foray into the gangster life show Americans rooted in Italian values. De Niro spends half the movie booming an American accent, and does a great job at becoming Robert De Niro. There is no part that does not have an emotional bite - the moment when Tom Hagen is kicked out is especially brutal. What a fantastic story.
TIL that the predecessor to Godfather II was not The Godfather, but Black Hawk Down.
Prompt: Write notes on Sion Sono's Cold Fish in the style of OrcaNinja.
Response: Cold Fish draws its power from a compelling first 45 minutes, Daisuke Miyagawa’s reptilian grin, and the real life horror of a crime committed with an unkillable Saito in charge. Sono pretends to introduce elements with exercises in metadrama and gore, but doesn’t do enough to follow through or critically comment on them. Saito’s sex dungeon standup is particularly wacky and doesn’t really make any sense. The acting and script are serviceable; the horror is not.
This one is better, but I think I missed the sex dungeon part of the movie.
Prompt: Using OrcaNinja's review of Love Exposure directed by Sion Sono, outline what he liked the most about the movie.
Response: The Lynch and Deep Throat allusions are a bit too knight’s move to imitate, yet somehow it works because of the absurd and epic nature of the rest of the film. Sono seems to lap in the stench of the perverts that enforce absurd, corrupt authority and gamblers that increase randomness in a universe already in chaos. The opening swordplay is awkward, funny, and actually impressive; those later qualities undoubtedly fit and carry through the entire picture. Murakami’s perversion is constructed to be righteous and somehow that right-wing, up-is-down mentality gets acclimated with the wild, long-winded universe created in Love Exposure. The gang oddly becomes a sympathetic group, even if it becomes hard to draw the line between outrage and amazement. By the second panty shot I knew the film’s game - to inundate viewers to the extremes of cult (late-)mid-aughts otaku culture with doses of “Romeo and Juliet”, Christianity, and the dysfunctional Japanese family - so to push up on the 4 hour mark (a more complex critique ignored in the rush to an ending is the nature of “trash pictures”) with such ease is uncanny. The film’s success in carrying introspective material in an exciting format while also not being shy about its subject matter, however, leads the two “quiet” scenes in the film to look incredibly weak.
This actually resembles my writing style the most! It’s too bad that it makes zero sense.
Prompt: Review Oppenheimer in the style of OrcaNinja
Response: Oppenheimer has a brutal first scene (all of that man’s films do, don’t they?) and follows through with an emotionally charged drama that distills one scientist’s guilt for his reluctant work, creating a bomb based on the theoretical results of his intellect. If there are moments where Pattinson falters, he’s surrounded by an excellent cast - a special mention for Cronenberg in the opposite role is absolutely necessary. Jan Brooks’ orgasms are some of the most effective I’ve seen in a long time. The black and white cinematography catches the weak Seattle sun but plays well in the dance hall’s dark shadows. There’s something far too romantic main character (and to crank that up, Nolan gives his POV the attention to detail) and that famously critical backstory is not given its true due.
This one started off okay and then went off the rails. I’m not quite sure where Jan Brooks came from…
Clearly I didn’t try my hardest on this section of the assignment. But this is ML/AI at it’s most fun!
4. Movie Reviews with a custom LLM
After all of the lazy work above, do you really think I would do this?
Closing thoughts
There are no academic conclusions to be made from this experiment. The role of the movie critic is not going away anytime soon, even if there is an uptick in people who use LLMs to generate outlines or content. Click-bait LLM-generated articles are currently pretty easy to spot. Even as these models get better, infusing a unique point of view using a model like this is an insanely hard task. For those with a bit of technical know-how, I highly recommend a side project like this which can help you get familiar with LLM tools while having some fun (the lame version: try harder than me, add it to your GitHub and talk about it in Data Scientist interviews).