A manga artist tries out Mantra Engine, an AI manga translation tool soon to be available on FANBOX!

The FANBOX team and the pixivision editorial team

Starting April 1st, 2022 (Fri), the AI manga translation tool Mantra Engine will be available for a fee to all creators with an existing pixivFANBOX plan. With Mantra Engine, all you have to do is upload the image files of your manga, after which the tool will take care of replacing the Japanese text with the English translation. Check the outcome, adjust the layout, and you're done!

Before the service's official launch on FANBOX, we asked mangaka Natsu Tadano to help us test Mantra Engine's AI translation with one of their manga. We asked them about their impressions, what they liked about it, and what still has space for improvement.

Sekino from Mantra Inc. and some members of the FANBOX team also joined the interview and told us all about the hardships involved in developing the tool and about some special features exclusively available on FANBOX.

＞＞Read Tadano's manga in English!＜＜

Natsu Tadano
Mangaka. Their debut work Yukuyuku Futari (Frontier Works) was first published on pixiv Comic on December 24, 2016. Other notable works include the cover art for the novel Koe no Wazaogi and Tsukanoma no Ichika (Kodansha, 3 volumes).
pixiv
FANBOX
Twitter

Mantra Inc.
With the goal to deliver manga across language barriers, the company is engaged in the research and development of machine translation technology for manga. Joining us this time is Mantra's business manager Sekino.
HP

About pixivFANBOX:

pixivFANBOX is a fan community where creators can nurture their creative activities thanks to the support of their followers. Creators can manage their own subscription service and make multiple plans, similar to a monthly fan club or paid blog. Learn more.

A tool that doesn't even need a tutorial

── Thank you for providing one of your manga for this test! What were your impressions after using Mantra Engine?

Tadano: I could say "good", but I'm afraid that doesn't cut it. (laughs) Whenever I try something new, I never read the instructions or watch the tutorial. In this case, I managed just fine... and that's quite something.

── You handled it by just looking at the screen.

Tadano: Yeah. Just a few clicks and it was done. You can pick an image file by dragging and dropping it into the tool, and the translation process is also straightforward: wait until the Translating... popup disappears, and there you have it! Your shiny English translation. Amazing!

The machine translation without any corrections. Most of the lines were translated correctly, but in the last frame the AI ended up misgendering the subject.

── It's nice when a tool is so intuitive.

Tadano: Right. Also, Mantra Engine comes with many more features than I thought. I imagined there'd be one to change the font size, but there were also 20 others that were completely unexpected!

Mantra Engine's editor. The original page is displayed on the left, and the page with the AI translation output is displayed on the right. Using the right sidebar, you can select the font and font size, along with many other editing options.

── What are some of these features?

Tadano: When translating from Japanese to English, the text is written horizontally instead of vertically, so the English dialogue often ends up sticking out of the speech balloons. Usually, I'd have to add a few line breaks, but here I can just add a new line in the edit box to see the changes reflected in the balloons.

There is also a "reverse translation" function, which translates lines from Japanese to English and back again to Japanese. In general, I was able to do most of the things I had set my mind to.

── I understand that the "reverse translation" function was developed especially for FANBOX creators.

Sekino: That's right. It was specially added to make things easier for those who aren't so confident in their English skills. Actually, Mantra Engine was developed with professional translators in mind, as a way for them to improve their translation efficiency. Pros can find translation errors on their own, so the "reverse translation" feature wasn't necessary. However, FANBOX pointed out that some creators may not be so self-assured about their English, so we added a "reverse translation" feature to let them check if the AI is doing its job correctly.

Tadano: And I'm so thankful for it. (laughs) My English isn't so good, so I wouldn't be confident enough to proofread the translation myself. If I had to do the reverse translation myself using a web-based translation service, I would have to copy and paste the text, input the revised version, and so on. This is sparing me the hassle.

── And when you correct the English, the reverse translation reflects the corrections in the Japanese text in real time, doesn't it?

Sekino: We worked very hard on that. (laughs)

── A wonderful effort by the people at Mantra Inc. Tadano, what feature was your favorite?

Tadano: It's a small thing, but I like the fact that the English text comes with a white border by default. It makes it easier to read even when it sticks out of the speech bubbles, and it's relieving not to have to make the text smaller to cram it inside the balloons.

Without any major corrections, all you need is one or two minutes per page.

── Did you have any setbacks?

Tadano: The Japanese for "wow!" comes up in the manga, and although it was translated properly by the AI, the reverse translation returned the equivalent of "woooooooow!" Not really a setback, I just thought it was funny. (laughs)

Also, in one of the lines, I used the word masayume (正夢, a "prophetic dream"), which the AI just couldn't get right. I tried looking the translation up myself, but I'm not good at English so it was a bit of a headscratcher.

Sekino: I guess there's no English word that perfectly corresponds to the Japanese masayume. You have to rephrase it to something like, "a dream come true."

The AI translation of masayume was just "dream."

── I see. Things can get challenging when the translation heavily relies on the context.

Tadano: I've been keeping an eye on the reverse translation while somehow translating the words in my head as well. The parts that sounded correct I left as they were, while I used Google Translate for all those lines that got the original meaning completely wrong.

── Translation engines are not perfect, so proofreading is a must. What was the hardest part of the translation process?

Tadano: The part where I don't speak English. (laughs) The dialogue in manga tends to be colloquial, so I'm not sure that the translated English text has the same nuances as the original Japanese. But this time my goal was only to get the message across, regardless of the nuances.

── How long did it take to get the work done?

Tadano: I spent quite a long time looking things up, but I guess it didn't take more than seven minutes per page. For pages where Mantra Engine did an especially good job, I'd say one to two minutes tops.

── That's quite a time saver compared to the time it'd take to get the translation done yourself. Is there any feature that you would like to see added to the tool?

Tadano: When you're done translating and select a file format to export your work, it takes a while for the Exporting in progress... popup to change to Exported! It's hard to tell whether the exporting process is working or if it stopped halfway, so it'd be nice to have some extra instructions, like This may take some time.

The tool offers a wide range of import and export formats, including PDF, PNG, JPEG, and TIFF.

Sekino: You should be getting an email notification when the exporting process is complete. Would it be better if we added an alert, such as You will be notified by e-mail when your file is ready?

Tadano: Yes, I think so.

Sekino: That's good to know. Thanks for the feedback!

Will the AI ever be able to translate handwritten text?

── Would you like to keep using Mantra Engine in the future?

Tadano: Fans outside Japan sometimes contact me to ask for permission to translate my manga. I'm happy to let them, but in the future, it'd be nice if I could translate them myself using Mantra Engine. The thing is, I usually write the dialogues by hand... and I guess machine translation doesn't support handwritten text, right?

Sekino: Unfortunately not... Mantra Engine's AI can only translate text that's recognized as Japanese. For character recognition, we have the AI learn existing Japanese fonts, but handwritten text is a different ball game.

Tadano: I see. Maybe if I use very neat handwriting, the AI will recognize it... (laughs) I guess I'd better use regular fonts whenever I plan to use machine translation. By the way, how many years do you think it'll take until machines can translate people's handwriting?

Sekino: Well, that's a tough question. (laughs) You see, Mantra Engine increases the accuracy of character recognition and translation by training its AI on a large amount of Japanese and English manga data. The problem is that there are not enough manga with handwritten dialogue, and since AI becomes more accurate the more data it learns, the scarcity of handwritten dialogue in circulation complicates things.

── You're talking about the so-called training data, right? AI accuracy can't be improved without huge amounts of training data.

Sekino: With existing fonts, you only need a few hundred types at most, but if you take into account individual writing habits, the number of different patterns becomes humongous.

On a different note, the onomatopoeia washa washa (わしゃわしゃ) comes up in Tadano's manga. We've been researching such handwritten onomatopoeia and sound effects, and have achieved about 70% accuracy in determining the location of the characters representing an SFX. Unfortunately, though, we still have a long way to go before we're able to translate them.

FANBOX: Incidentally, many readers outside Japan prefer the SFX in their manga to remain in Japanese. They feel that onomatopoeia and sound effects are part of the picture, and they like their original look.

Tadano: For us Japanese people it's the same with western comics, isn't it? Even if we don't get what the sound effects mean, they help us visualize the overall atmosphere of the scene. Sometimes creators use different types of lettering to match the nuances of the sound, so maybe the SFX should be left untouched.

── What should someone do if they want to translate handwritten text or SFX?

Sekino: It seems like most people like their handwritten onomatopoeia, so it may be a good idea to keep the original and add a translation on the side. To do so, click the "+" button to generate a box where you can freely add your own text. Keep in mind that handwritten text and SFX will not be translated automatically, so you'll need to enter the English directly and adjust the size and tilt to place it.

Sekino: If you want to translate handwritten text by replacing it instead, we recommend erasing the text before importing your image file to Mantra Engine and re-typing it using a vertical Japanese font ー that way the AI can translate it.

Need to translate a few pages right away? No problem!

── Besides reverse translation, are there any other special features available only on FANBOX?

FANBOX: When you use Mantra Engine on FANBOX, you will be given account privileges for two collaborators (in addition to the creators themselves) who can help you with the translation work. This allows creators who are not confident in their English or who are busy with their creative work to get the support of their fans and friends.

── Not to see the glass half empty, but does the account come with sturdy security features?

Sekino: Collaborators have limited access and can't upload or download images.

── So you can have your collaborators help out with the translation without handing over the entire project.

Tadano: That sounds like a dream feature! (laughs)

Sekino: Thank you! (laughs) We have taken these measures to discourage piracy, to name one issue, but also to prevent collaborators from translating someone's work without permission and taking all the profit. If Mantra Engine manages to make translation easier, it'll let us take a step forward to solve such fundamental problems.

FANBOX: Works translated in Mantra Engine can be freely used and sold outside of FANBOX, for example on social media like pixiv and Twitter, as well as made available for paid download. Of course, we'd appreciate it if you posted your translated work to FANBOX! (laughs)

── Let's talk about price. How much does the service cost?

FANBOX: Mantra Engine is available for a monthly fee under the Creator's Privileges system. Two plans are available with different page limits: 3,000 JPY (approx. 30 USD) for up to 16 pages, and 5,500 JPY (approx. 55 USD) for up to 32 pages. How does that sound?

Tadano: It really depends on how many English-speaking fans you have, but the price sounds reasonable for someone with a decent international following.

FANBOX: In April 2022, the first month of the service, we're offering a cashback campaign that lets creators take advantage of the 16-page plan for only 800 JPY (approx. 8 USD). After purchasing the monthly plan for 3,000 JPY, FANBOX will refund you 2,200 JPY the following month. It's a good deal to test the waters with your international public!

Tadano: I see. If creators find out that people are interested in their translated work, they might want to keep using the service.

FANBOX: If you outsource the job to a professional translation company, in addition to paying a translation fee you may also be charged for the typesetting. We don’t mean to brag, but considering the final price per page, Mantra Engine (which also offers typesetting features) is quite convenient...

Also, translation agencies often have a minimum number of pages required to take on the job, and delivery times can be quite drawn-out.

Tadano: Yeah, you can't really ask a translation agency to work on a short manga you wrote on a whim to publish on FANBOX, pixiv, or Twitter.

Sekino: With Mantra Engine, you don't have to worry about the number of pages. If our service succeeds in creating new demand, we'll be very happy developers.

── Thank you for your valuable insights!

＞＞Learn more about Mantra Engine!＜＜

＞＞Read the translation of Tadano's manga!＜＜

Limited cash-back campaign for April 2022!

To celebrate the launch of Mantra Engine on FANBOX under the Creator's Privileges system, we're offering a special cashback of 2,200 JPY to all those who sign up for the monthly 16-page plan.

To access the campaign, pre-register between March 18th, 2022 (Fri) at 12:00 a.m. JST, until March 31st, 2022 (Thu) at 11:59 p.m. JST. Don't miss the valuable chance to try out Mantra Engine for only 800 yen for 16 pages of translation!

Promotion period:

From April 1st, 2022 (Fri) 11:00 a.m JST. until April 30th, 2022 (Sat).

* Sign up in advance between March 18th, 2022 (Fri) at 12:00 a.m. JST, until March 31st, 2022 (Thu) at 11:59 p.m. JST.

* To apply for the promotion you must create a plan on FANBOX beforehand.

* The promotion is available to 500 eligible users, on a first-come-first-served basis.

＞＞Sign up here!＜＜

Three features of the FANBOX✖️Mantra Engine collaboration

Mantra Engine is a cloud service that uses AI to automatically detect speech bubbles and dialogue from manga pages and translates them into other languages using AI translation. Here are three reasons why it's so convenient!

① Translate one page in as little as 30 seconds! Simply upload an image and get the English translation you need.

Mantra Engine is super easy to use. Simply upload two versions of your manga: one with the Japanese text and one without. The AI automatically detects the Japanese in the speech bubbles and translates the dialogues in as little as 30 seconds per page! The tool comes with a variety of fonts, text sizes, and colors. And everything can be done in your browser!

② Host up to two collaborators. Get your friends and fans to help with the translation work!

You can designate up to two collaborators (in addition to the creator themselves) and grant them account privileges. They will not be able to upload and download the image data, so you don't have to worry about security!

③ If you're not confident in your English skills, the "reverse translation" feature is there for you.

A special reverse translation feature is available exclusively on FANBOX! It translates lines from Japanese to English and back again to Japanese, pointing out any mistranslations and lending a helping hand to all those creators whose English is a little rusty. How convenient is that?

＞＞Learn more about Mantra Engine!＜＜