M2M-Gen: Background Music Generation for Japanese Manga using Large Language Models

Input for Page-level Caption Conversion
*Dialogues of the Scene (Output of Step 1)*
The dialogs in the image are as follows
PAGE 23 エリー: うっ！？
PAGE 23 エリー: ドクン
PAGE 23 エリー: この嫌な感じ.....
PAGE 23 エリー: ドクン
PAGE 23 エリー: さっきの.......
PAGE 23 エリー: ドクン
...
...
PAGE 27 エリー: そこには不安が横たわっていた.....
これで奴を葬り去ることができたんだろうか......
PAGE 27 Others: 大佐っ！
葉巻やめて下さい
PAGE 27 大佐: いいじゃねえかよ
かたいこというない
PAGE 27 エリー: 異形の魂を覗きこんだ恐怖がまだ胸にのこってる.....
*Emotion Recognition of the Scene (Output of Step 2)*
The faces have the following emotions: 57% neutral, 13% surprise, 13% fear.
*Scene-Level Caption Conversion (Output of Step 3)*
The music description for the entire scene is: A suspenseful orchestral track with dark, pressing strings, and intense percussions with occasional pauses for shock, amplifying a sensation of fear and urgency.

Please Generate a music description for page 23 in the scene

Output for Page-level Caption Conversion (Output of Step 4)
Page 1 Caption: A distressing symphony with a rapid tempo, overwhelming the tense atmosphere with drastic violin strokes and nerve-wracking drum beats, creating a sense of urgency and danger.
Page 2 Caption: A high-intensity orchestral track with pounding percussions and rising strings characterizing the escalating tension and ensuing chaos. The music oscillates between rapid, alarming sequences and abrupt, suspenseful pauses that signify pivotal moments of challenge and decision-making. A continuous undercurrent of unease simmers throughout, signifying the looming dread and high-stakes pressure.

Input
Page 1 Description The manga panels depict an intense battle scene, with dramatic action lines and explosive effects. A character is seen piloting a sophisticated machine or weapon, with various expressions of determination and concern. Dialogue in speech bubbles suggests strategic communications and thoughts in the midst of the conflict.
Page 2 Description The manga page features an intense aerial battle scene with dramatic expressions and dialogue. Characters are seen communicating mid-battle, likely discussing strategies, while a large aircraft dominates the middle panels, illustrating high-speed maneuvers through the sky. The page is marked by dynamic action lines and explosive sound effects that emphasize the intensity of the conflict.

Output
Page 1 Caption: A fast-paced, high energy soundtrack featuring booming percussion to symbolize the explosive action and blaring brass to highlight the tension of the battle. Incorporating intermittent electronic synths to reflect the sophistication of the machine and quiet, somber strings during strategy discussions to bring out the characters' concern and determination.
Page 2 Caption: A fast-paced and intense orchestral score with soaring strings and urgent percussion, embodying the high-speed aerial combat. Intermittent heroic brass melodies symbolize strategic talks amidst the battle, while high rhythmic staccatos illustrate the dynamism of the scene. The music concludes with a powerful crescendo, showcasing the exploding conflict.

Input for Page-level Caption Conversion
*Dialogues of the Scene (Output of Step 1)*
The dialogs in the image are as follows PAGE 8 有森杏子: 深見くん!?
PAGE 8 有森杏子: あの死んじゃった深見くん!?
PAGE 8 有森杏子: 死んだ人間がなんでそこに………
PAGE 8 有森杏子: ねえったら！
PAGE 8 有森杏子: 葉月！
...
...
PAGE 9 有森杏子: 深見くんって成績はいいけどその……弱い子で
PAGE 9 有森たつろー: ーわかるよ点数だけとっていればぜんぶうまくいくと思ってる連中だろそういう落ちこんだことのないやつが怖いんだ
PAGE 9 有森たつろー: 死に方も過激だね……
PAGE 9 有森たつろー: やけになってよじのぼった鉄塔に落雷なんて……
PAGE 9 有森たつろー: そりゃ葉月って子もうなされるぜ
PAGE 9 有森杏子: 彼女にはそんなこと言っちゃだめよデリケートな子なんだから
*Emotion Recognition of the Scene (Output of Step 2)*
The faces have the following emotions: 30% neutral, 20% surprise, 20% happiness.
*Scene-Level Caption Conversion (Output of Step 3)*
An eerie symphony with haunting piano notes, punctuated by the sudden gasps of violin strings, embodying a chilling blend of surprise, dread, and unease.

Please Generate a music description for page 8 in the scene

Output for Page-level Caption Conversion (Output of Step 4)
Page 1 Caption: The eerie symphony takes a menacing turn with an unsettling piano melody that underscores the shock and dread in the dialogue, while sudden outbursts from the violin strings highlight the moments of surprise, creating an atmosphere of growing unease.
Page 2 Caption: A melancholic piano melody imbued with chilling dread for the haunting recollection, followed by sudden eerie trembles of violin strings enhancing the surprising revelation and lingering unease.

Input
Page 1 Description In the manga pages, a young girl appears to be in distress as she receives shocking news over the phone. The grayscale artwork captures intense expressions and dramatic atmospheric effects, conveying a sense of urgency and fear as the girl confronts an unknown danger.
Page 2 Description The image depicts two scenes from a manga book: one page shows a girl in a dramatic moment, seemingly in distress and being struck by lightning; the other page involves a conversation between characters, addressing a critical situation while they are inside a car, with a focus on their expressions and reactions. The atmosphere appears intense and emotionally charged

Output
Page 1 Caption: The music is a crescendo of high-pitched strings, grated with haunting minor melodies conveying an atmosphere of tension and shock. A sparsely played piano underscores the fear, adding a somber tone, while subtly escalating percussions induce the sense of urgency and anticipated danger.
Page 2 Caption: The music should be intense and electrifying, signifying the emotional turmoil. Begin with sharp, aggressive strings that depicts the strike of lightning. Transition into a somber, dramatic melody with slow-paced piano notes and solemn violins to illustrate the tense conversation in the car. Keep the music dramatic and suspenseful throughout

Input for Page-level Caption Conversion
*Dialogues of the Scene (Output of Step 1)*
The dialogs in the image are as follows
PAGE 75 ナギ: ぽっちゃま？
PAGE 75 ダニエル: なんでもない！
ナギ
お茶をいれておくれ
PAGE 75 ナギ: ぽつちゃま
覚えてなさるか
PAGE 75 ナギ: あの頃
PAGE 75 ナギ: 花の野
花の野
どこまでも広い
ヴィコント家の領地
...
...
ちょっとコツを覚えれぱあとはカンタン
PAGE 81 ナギ: ほらこうやって
PAGE 81 ナギ: これは左手
*Emotion Recognition of the Scene (Output of Step 2)*
The faces have the following emotions: 39% neutral, 29% happiness, 16% sadness.
*Scene-Level Caption Conversion (Output of Step 3)*
A lively dance of piano notes playfully intertwining with a gentle violin melody, accentuating the moments of lighthearted joking and hide-and-seek game. The music takes a mysterious turn, hinting at the presence of deeper thoughts beneath the surface, as subtle, low violin strings produce tension in the background. Please Generate a music description for page 80 in the scene

Output for Page-level Caption Conversion (Output of Step 4)
Page 1 Caption: A slower, somewhat melancholic piano tune, paired with the occasional drop of silence, reflecting pensive moods and a tint of unease.
Page 2 Caption: A cheerful piano tune picks up the pace as the characters discover the marionette, guiding the mood of joyful exploration and mutual teaching, with strings subtly underscoring the moments of shared happiness.

Input
Page 1 Description In this manga scene, two characters engage in a conversation inside a sophisticated room adorned with roses and elegant furniture. One character seems concerned while the other appears confident and possibly scheming. The setting also includes an exterior view of a large, ornate building, suggesting a storyline filled with drama and intrigue
Page 2 Description The manga pages depict a scene in which two characters interact inside an elegant building with intricate architecture. One character seems to be showing the other a marionette or puppet, leading to a display of its movements and mechanics, as evident from the close-up interactions and expressions.'

Output
Page 1 Caption: A classic orchestral piece encompassing elements of suspense with low, brooding violins and sudden emphasis on cello notes to evoke the scheming character's confidence. Subtle harp strings conjure the elegance of the room, whereas, low trills from a piano illustrate the concern within the conversation
Page 2 Caption: The music is a genteel blend of orchestral and chamber music, intertwining the sophistication of violin and the playful tickling of the piano. The melody floats like a drifting marionette, its elegance mirroring the intricacy of the architecture, marked by moments of surprise and wonder to reflect the puppet's movements

Input for Page-level Caption Conversion
*Dialogues of the Scene (Output of Step 1)*
The dialogs in the image are as follows
PAGE 14 山朱秋緒: おいお前酒くせぇぞ？
PAGE 14 北ケ谷冬華: オイオイ朝から飲んできたのかよー
PAGE 14 藤原成実: フツーにごはん食べてきただけだよぅ
PAGE 14 藤原成実: うん
PAGE 14 藤原成実: ええっ！？
PAGE 14 山朱秋緒: 藤原！今すぐにバイト行こう！
PAGE 14 藤原成実: なに言ってんのまだ学校にきたばっか…
...
...
PAGE 15 藤原成実: 家一軒とその近所の清掃と消毒を頼みたいんスけどおー
PAGE 15 北ケ谷冬華: えっ
PAGE 15 北ケ谷冬華: 近所入れると百万ぐらいかかるんスか？
*Emotion Recognition of the Scene (Output of Step 2)*
The faces have the following emotions: 52% neutral, 14% happiness, 14% anger.
*Scene-Level Caption Conversion (Output of Step 3)*
A lighthearted, conversational jazz ensemble piece, featuring a playful piano theme, a grooving stand-up bass line and subtle percussion, reflecting the everyday chatter and occasional disagreement. Please Generate a music description for page 14 in the scene

Output for Page-level Caption Conversion (Output of Step 4)
Page 1 Caption: A playful yet casual jazz ensemble piece that matches the rhythm of the characters’ lively conversation and arguments, with the piano theme underscoring the mood of the everyday dialogues and a subtly grooving bass line capturing the occasional surprises.
Page 2 Caption: A playful jazz ensemble continues, the piano theme becoming more involved and the stand-up bass line grooving, mirroring the heated decision-making and hustle of the group, as they discuss how to deal with the issue at hand.

Input
Page 1 Description The manga pages depict a humorous scene where a group of characters are discussing and experimenting with food. One character appears astonished, while another holds a plate of food and starts explaining something. The other characters show various reactions, ranging from confusion to curiosity, while engaging in animated conversation.
Page 2 Description The image depicts two pages from a manga book with several characters interacting. On the right page, there's a large mushroom growing on top of a house, surprising the onlookers. The left page shows multiple characters engaging in a conversation, with one character lying on the ground, seemingly exhausted or unwell

Output
Page 1 Caption: A lively and upbeat jazz piece, filled with plucky pizzicato strings and playful piano notes. This piece should include sudden upward shifts to highlight the surprising moments and a jovial rhythm to keep up the humorous dialogue and exaggerated reactions. The overall tone is light and whimsical to suit the atmosphere of comedy and confusion.
Page 2 Caption: A mysterious and whimsical score featuring light orchestral instruments with a lingering suspense. The music transitions from an airy, somber tone for the fallen character to a bewilderment-filled crescendo upon the reveal of the mushroom building. Elements of surprise intertwine with the music's organic flow.

Input for Page-level Caption Conversion
*Dialogues of the Scene (Output of Step 1)*
The dialogs in the image are as follows
PAGE 4 理事長: フフやんちゃね
PAGE 4 理事長: まあいいわ…
PAGE 4 理事長: 嘩形 弾
15歳…
PAGE 4 理事長: 高等少年院より編入…か
PAGE 4 理事長: 安心なさい
PAGE 4 理事長: 外でどんな悪きをして来ようと学園内部は規律の下皆平等よ
PAGE 4 高田一成: 当学園はどんな問題児でも受け入れます
PAGE 4 高田一成: どんな人聞でも…
PAGE 4 高田一成: 「人聞力」が高けれぱ…
...
...
PAGE 8 嘩形弾: １年13組ねぇ…
PAGE 8 嘩形弾: お
PAGE 8 嘩形弾: なァ おい１年13組って…
*Emotion Recognition of the Scene (Output of Step 2)*
The faces have the following emotions: 30% happiness, 21% anger, 14% surprise, 14% fear, 14% neutral.
*Scene-Level Caption Conversion (Output of Step 3)*
A confrontational orchestral piece filled with quickly escalating tempo, highlighting the heated exchange between characters. The sudden burst of brass instruments signifies the character's rebellious defiance, intertwining with discordant chants that underline their tumultuous dialogue and clashing personalities. The rising tension culminates into a climactic crescendo, inducing a sense of chaos and lawlessness, perfectly encapsulating the scene's conflict and defiance.
Please Generate a music description for page 7 in the scene

Output for Page-level Caption Conversion (Output of Step 4)
Page 1 Caption: As page 7 of this scene harbours conversations laced with confusion and multiple emotions, the music description here would be a blend of softer orchestral tones interspersed with hints of discordant choir chants, subtly hinting at the evolving tension within the group.
Page 2 Caption: On page 8 of the scene, the orchestral music continues to provide suspense, but lightens slightly with traces of humor, reflecting the characters' witty banter and light-hearted comments. Despite the abrupt shift in tone, the underlying tension is subtly represented in the occasional discordant choir chants.

Input
Page 1 Description In this manga scene, several characters are depicted in various intense and dynamic moments. The central panel on the left page shows a character in a cloak walking away with dramatic effect lines emanating from them. Other panels depict close-ups of characters' faces displaying strong emotions, including anger and determination, engaging in discussions and confrontations."
Page 2 Description This manga page features several panels depicting two characters in a setting with tall buildings. One character appears intrigued or surprised, while another walks with casual confidence. The right page shows a detailed, expansive view of an urban cityscape from a high vantage point.

Output
Page 1 Caption: The music should create palpable tension with thunderous, high-tempo drums and sharp, high-pitched violin streaks, capturing the intense emotions sprawling across the page. Occasional low, mysterious undertones should hint at unknown elements associated with the cloaked character. The melody should weave a sense of urgency, determination, and conflict.
Page 2 Caption: A smooth jazz track with a confident walking bassline and curious saxophone solos. The music incorporates urban elements such as subway ambience, presenting a panoramic auditory perspective of the city, and gradually intensifying to match the grandeur of the high-view cityscape.

Input for Page-level Caption Conversion
*Dialogues of the Scene (Output of Step 1)*
The dialogs in the image are as follows
PAGE 3 男A: 例のものです　どうぞ
PAGE 3 男A: こちらにサインを
PAGE 3 黒井戸: はい
PAGE 3 黒井戸: いつも御協力いただき感謝します
PAGE 3 男A: いやぁ最近は皆さん臓器提供に理解がありますからね
PAGE 3 黒井戸: 全く有り難い話です
PAGE 3 黒井戸: では私はこれで
PAGE 3 黒井戸: 今日はいい天気だ
PAGE 4 黒井戸: ほおこれは
PAGE 4 郷保: ！！
...
...
PAGE 6 黒井戸: つまりあなたがこの目を入れて見ることにより
PAGE 6 黒井戸: この人の見た物をあなた自身の目として見　感じる事ができるのです
PAGE 6 郷保: "入れる"？
*Emotion Recognition of the Scene (Output of Step 2)*
The faces have the following emotions: 43% neutral, 28% happiness, 11% fear.
*Scene-Level Caption Conversion (Output of Step 3)*
A gentle piano melody that evokes a sense of hope and neutrality, reflecting the polite and business-like conversation between the characters. The faint pitter-patter of light percussion subtly sets the rhythm, symbolic of the routine nature of the interaction. There are no significant string undertones as the suspense hasn't built yet at this stage.
Please Generate a music description for page 3 in the scene

Output for Page-level Caption Conversion (Output of Step 4)
Page 1 Caption: A gentle piano melody setting a neutral and calm tone, harmonizing the peaceful conversation happening, subtly underscored by suspenseful string undertones reflecting the hidden implications of the dialogue.
Page 2 Caption: A gentle piano melody that creates a sense of hopefulness, coupled with light percussion to reflect the friendly, neutral dialogue, subtly interspersed with faint strings hinting at underlying tension and the unknown.

Input
Page 1 DescriptionThe manga page shows a sequence where a person is jogging outside and then entering a building. Inside the building, a doctor or scientist with dark hair and glasses is shown holding a clipboard and overseeing a bizarre experiment involving floating eyeballs in a container. Another character watches the jogger from behind a tree, holding an object (perhaps a pen or a device)
Page 2 Description The manga pages depict a conversation between two male characters, one of whom seems to be an artist showing his painting to the other. The expressions range from surprise to delight as they discuss the artwork and its abstract nature. The interaction appears lighthearted, occurring in both an indoor and outdoor setting, possibly near a park.

Output
Page 1 Caption: An energetic, upbeat melody mimics the energy of the jogger, featuring light percussion and synthesisers. As the scene transitions indoors, the rhythm graduate into a mysterious and suspenseful orchestration with a strong but eerie synth line accentuated by soft, creeping strings. Muted brass instruments reflect the peculiar experiment, while stealthy, shadowy tones mirror the character observing in secrecy
Page 2 Caption: A light, playful melody on a piano with whimsical undertones perfectly encapsulates their interaction. Strings provide depth and evokes the abstract art discussion. Soft nature sounds, like a gentle breeze rustling leaves or birdsong, paint aural representation of the park setting.