About

AI Animation &
AI tools Redesign

A binge study & redesign of 15+ AI tools based on Human-Centered AI and XAI theory

72 hours for an AI-powered animation from scratch to launch
15+ exisiting Generative AI products audited & redesigned

Aired in 2023

UX Audit

UX Redesign

Storytelling

Visual Design

Project Goals

Bring the yearly-celebrated event SloanFronts creativity show to the next level

Design Goals

Design an anime for the show like movie storytelling in 72 hours from scratch, ideally fully powered by AI

OUTCOMES

PROCESS

DISCUSSION

IMPACTS

SIGNIFICANCE

There was a growing need for Teaching Assistants at non-design schools to understand essential media production knowledge. I initiated a UX project that quickly demonstrated AI-powered production for academic use.

From a UX design perspective, the project sparked multiple discussions on AI-enabled function design.

It was also a short-term, intensive learning experience for me, where I grasped the rapid iteration pace of AI-powered products.

TEAMS

  • Sean Huang (Me) - Design Lead & Researcher

  • UX Audit | UX Research | Product Redesign | Production

  • Ben Ryan Shields - Producer & Supervisor

  • Yvette Kong - Supervisor

BACKGROUND

As a Teaching Assistant for the Creative Industries 23-24 class at MIT, I introduced a new topic: the impact of AI-Generated Content (AIGC) on human creativity.

To explore whether Gen-AI could overshadow human creativity and production, I set myself an ultimate challenge: leveraging AI tools to create an anime for the 2023 SloanFronts design show in just 72 hours.

This experiment aimed to test the extent to which AI tools can truly replicate human capabilities.

TOOLS & METHODS

Research & Analytics

Desktop Research | UX Audit | Personas

  • Figma | Product Hunt

Design & Testing

User flow | Storyboarding | Fast - Prototyping | Animation

  • Figma | Adobe Creative Cloud (Photoshop

, After Effects, Illustrator)

OUTCOMES

4-min AI -Powered Animation

UX Auditing Summary

based on a typical animation production process

PROCESS

The year 2023 marks a pivotal moment for MIT as it integrates AI technology into its pedagogical environments, with both empowering opportunities and significant challenges.

As a Teaching Assistant for the Creative Industries class, I was given the challenge and opportunity to teach students about AI-Generative Content (AIGC) through a hands-on project:

creating an AI-powered animation in 72 hours for the annual creativity show, showcasing the culmination of the class.

Problems

Challenge 1 - Timing

72 hours for an Anime literally from scratch sounds like a mission impossible, given the overall procedure (storyboarding, character design, sound design, audio making, graphic design, dramatic motion work, video editing, etc.).


Challenge 2- Content

  • Giving an exact prompt with the most concise and accurate description to AI tools is hard due to translation nuances and the depth of description details;

  • For cultural differences, taking the pun suggested by AI tools might be risky, since some punches only work for specific people;

  • Before this event, I knew little about audio-making and voiceover generation.


Challenge 3 - Delivery

  • I have limited sense of how native-speakers (the majority of the audience) could vibe with the verbiage/tone suggested by AI.


Solutions

Regardless of the ambiguity, I decided to have a fast UX audit on the main AI tools for each phase of creation and stick to the relatively most intuitive option with shorter learning curve based on UX Auditing criteria:

Accessibility

e.g., Tool tips for laypersons to understand each piece of UI

Explainability

i.e., results & algorithms explainable to users

navigation

of Info Architecture ("IA") (for beginners, a comprehensive but intuitive IA helps save lots of time)

page Speed

it determines possibilities of multitasking


User Research & Persona

I learned Five different XAI needs and the importance of being mindful of human cognitive biases.

As a beginner myself, I decided to delve into the persona of a beginner to Gen-AI tools who is more desperate than the others for precise and accurate AIGC outcomes with a pursuit for a less steep learning curve.

🔼 A diagram of different audiences for a given prediction and explanation by Meg Kurdziolek (2022)

User Flow

Despite that most current AI tools are not positioned as consumer products thus making it too early to discuss UX design for such products, I still found two points where UX & UI design can very essentially enhance the user journey:

Space for explainability 1

Before users input prompts to AI, i.e., the stage before the blackbox system. This phase entails clean, intuitive and instructional UIs to assume that even if a beginner without watching tutorials can operate the tools smoothly.

Space for explainability 2

After the AIGC results are provided. In this phase, users will make decisions quickly whether to accept the results, modify prompts for regeneration or abandon, which requires the system to provide feedback on users’ regarding their wrong inputs, better alternatives, etc..


AI Tools Redesign

I binge studied 10+ AI tools covering the phase of pre-production, production and post-production by:

  • desktop research on AI product reviews;

  • picking the tools most suitable for my case and walking through all essential tutorials;

  • and mapping out their key pros and cons (see the summary below)

    1. Midjourney

    Before

    Midjourney is powerful but more enclosed than other image Gen-AI tools. Regional modification is allowed for a generated image but not based on new prompts.

    1. Midjourney

    After

    In my redesign, users are able to customize regional regeneration upon their new prompts

    1. Runway

    2.1 Before

    Runway's multimedia AI-generated content offers impressive features, yet some, like the image-to-image function, are hindered by a counterintuitive user interface. For instance, a long list of text-based descriptions for visual styles can be overwhelming and unclear, leaving users confused about the visual content.

    1. Runway

    2.1 After

    I've redesigned this section to include visual thumbnails, making artistic terms more accessible and reducing cognitive load for users.

    1. Runway

    2.2 Before

    The current design allows for some customization but lacks clear guidance. For example, first-time users unfamiliar with the term 'Seed' find no helpful tooltips in the current design. Additionally, there's no way for users to comment on or provide feedback about the generated results.

    When inputting specific keywords like 'Junji Ito,' which typically represents a black and white comic line art style, the system fails to explain why the results may deviate from expectations.

    1. Runway

    2.2 After

    I understand there might be technical or other reasons why current AI tools don't explain their results for each piece. Nevertheless, I've redesigned the interface to include a speech bubble pop-up, helping users understand which part of their prompts might have led to unexpected content generation.

    I've also provided clearer examples of negative prompts to inspire and guide users.

    Tooltips have been added to key sections like 'Seed' to assist newcomers in understanding the finer details.

    1. Eleven Labs

    3.1 Before

    Eleven Labs has made significant strides since its initial release, yet there's always room for improvement.

    For example, the system introduced sliders for users to customize voice generation and set safety ranges to prevent inappropriate outputs.

    1. Eleven Labs

    3.1 After

    While the sliders are a good feature, I've enhanced them with preset scenario tooltips (e.g., optimal settings for audiobooks/marketing) and an official voice preview function. This way, users can more easily configure their ideal voices without constant adjustments.

    1. Eleven Labs

    3.2 Before

    In the voice generation section, the customization options were limited. Moreover, users sometimes struggled to gauge how sample voices would sound with exaggerated expressions if the sample text didn't include specific voice types.

    1. Eleven Labs

    3.2 After

    In my redesign, I've enabled users to experiment with various text-to-speech options. Additionally, I've integrated the Speech Synthesis Markup Language (SSML) as an option, allowing users to precisely achieve the vocal effects they desire.

    1. D-ID

    4.1 Before

    A significant challenge in creating smooth animation is lip syncing. Current AI tools, like D-ID, only allow for speaking human figures with frontal facing, which is highly limiting for animation creation. This means that if the character is non-human or posed in a side profile, these AI tools won't produce satisfactory results.

    Additionally, the guides for D-ID's lip syncing feature can be confusing, especially when they list several criteria in text form, such as measurements or numerical calculations (e.g., understanding the size of 200x200 pixels by sight).

    1. D-ID

    4.1 After

    In my redesign, users can use a dummy as a template to proportionally upload, crop, and resize their character images.

    1. D-ID

    4.1 After (Cont'd)

    For images that aren't exactly frontal facing, users can pinpoint the characters' facial features to assist the AI in mapping the face asymmetrically, resulting in more accurate figures.

    1. D-ID

    4.2 Before

    When selecting voices in D-ID, users were presented with a scrolling list of all available voices without detailed attributes, forcing them to essentially guess which voice would best suit their needs.

    1. D-ID

    4.2 After

    In my redesign, I've added attribute tags to each voice, allowing users to more quickly and accurately find their ideal voice.

    1. LALAMU

    Before

    Additionally, the guides for D-ID's lip syncing feature can be confusing, especially when they list several criteria in text form, such as measurements or numerical calculations (e.g., understanding the size of 200x200 pixels by sight).

    1. LALAMU

    Before (Cont'd)

    Additionally, the guides for D-ID's lip syncing feature can be confusing, especially when they list several criteria in text form, such as measurements or numerical calculations (e.g., understanding the size of 200x200 pixels by sight).

    1. LALAMU

    After

    For images that aren't exactly frontal facing, users can pinpoint the characters' facial features to assist the AI in mapping the face asymmetrically, resulting in more accurate figures.

Outcome Summary

user Case Report ✅

Looking into the real time commitment and my anticipation, I found the several points when I could stronlgy feel what the current AI tools were not able to cover:

Making an impressive and creative story script (not a mediocre or cliche one);

Create cohesive and uniformed characters from beginning to the end;

Doing dramatic or exaggerating animations without sacrificing the high-pixel visual quality;

Smooth lip-syncing for multi-dimensional figures.

Discussion on future Gen-AI feature Design ✅

As we explore various tools across different production phases, it's becoming clear that the future of UX design will heavily focus on explaining AI to users, emphasizing explainability.

I suggested key elements of effective UX/UI design for a generative AI tool as follows:

Regional modification Allowed

Hard!!!

Regional, more refined sub-scale alterations and modifications based on prompts

Feedback System

Intermediate

Help improve generation quality by enabling instant feedback on each generation

Presets & Templates

Easy

The most generally used preset parameters and templates, scenarios for prompts, etc.

Clear Previews

Easy

  • Images - visual styles

  • Audio - voice tones, emphases, expressions, etc.

  • Video - movement directions, etc.

Intuitive Guides

Easy

Tips for crafting effective prompts to improve generation efficiency, essential guides for beginners to understand the finer details

Alerts On Privacy

Easy

Prevent misuse of private information in customized model training, such as commercial exploitation or accidental disclosure of confidential information

IMPACTS 🎉

Sloan Design Show 🎉 🎉

Brought to the next level with creativity & awareness!

redesign concepts accepted 🤩

Some of the concepts were validated in the lastest versions of the tools!

Thanks for reading!
See More Projects 🔽