What tools are needed to build a meeting summarizer?

Key tools include Python for programming, Streamlit for building the web interface, and the Assembly AI API for speech-to-text and NLP features. Libraries like 'requests' and 'pandas' are also essential.

How does Assembly AI process audio files?

Assembly AI provides an API that accepts audio files. It transcribes the speech, can identify categories, and automatically segment the audio into chapters with summaries and gist points, all of which can be accessed via API responses.

How can I make chapter summaries clickable to jump to that part of the audio?

You can use Streamlit's session state and callback functions. By storing the start time of a chapter in the session state when a button is clicked, you can dynamically update the audio player's start time.

What information is extracted from the meeting audio?

The application extracts main themes (IAB categories) and provides detailed summaries for automatically generated chapters. Each chapter includes a longer summary, a headline, a gist, and specific start/end timestamps.

How can the user interact with the generated summaries?

Users can upload an audio file, see the main themes, and then browse through chapter summaries. Importantly, each chapter summary is linked to a button that, when clicked, will jump the audio player to the beginning of that specific chapter.

Key Moments

Auto-generating meeting notes with Python

AssemblyAI

People & Blogs4 min read25 min video

Mar 15, 2022|6,014 views|117|14

Save to Pod

Key Moments

TL;DR

Automate meeting summaries and chapterization from audio using Python and AssemblyAI.

Key Insights

Builds a Streamlit app to automatically summarize meetings from audio recordings.

Uploads audio, displays it, and extracts main themes and categorized chapters.

Utilizes AssemblyAI API for speech-to-text, summarization, and chapter generation.

Code includes a Python file for API communication and a Streamlit app file.

Features include audio playback, theme listing, and chapter summaries with clickable timestamps.

Session state and callback functions in Streamlit enable dynamic audio player control.

INTRODUCTION TO THE APPLICATION'S CAPABILITIES

This tutorial demonstrates how to create a Python application using Streamlit to automatically summarize meetings from audio recordings. The primary goal is to provide users with a quick overview of key discussion points, especially for long meetings. The application allows users to upload an audio file, view it, identify main themes discussed, and access automatically generated chapters with summaries. Each chapter can be played back individually, allowing users to jump directly to relevant sections of the meeting.

STREAMLIT APPLICATION STRUCTURE AND FILE UPLOAD

The application starts with a basic Streamlit structure. It prompts the user to upload an audio file. If a file is successfully uploaded, Streamlit displays the audio player. Initially, the player's start time is set to zero, but this will be dynamically updated later. The project is organized into two Python files: one for handling communication with the AssemblyAI API and another for configuration, such as storing the API token.

ASSEMBLYAI API INTEGRATION FOR TRANSCRIPTION AND ANALYSIS

Communication with AssemblyAI's API is handled by a dedicated Python file using the `requests` library. This file includes functions to upload audio files and initiate transcription. The API requests include headers with an authentication token and content type. Crucially, the transcription request can be configured to enable features like IAB categories (topic detection) and auto-chapters, which segment the audio into logical parts with summaries and headlines.

PROCESSING TRANSCRIPTION RESULTS AND STATUS CHECKING

After uploading the audio and requesting transcription, the application needs to poll the AssemblyAI API to check the status of the process. A loop continues until the transcription status is 'completed'. The API response provides a unique polling endpoint. By making GET requests to this endpoint, the application retrieves updates on the transcription progress and eventually the final results, including categories and chapter data.

DISPLAYING MEETING SUMMARIES AND CHAPTERS

Once the transcription is complete, the application displays the extracted information. Main themes, identified through IAB categories, are presented in an expandable section to keep the interface clean. The chapter summaries are organized into a pandas DataFrame for manageability. Each chapter includes a gist (short summary), a longer summary, and start/end timestamps. These timestamps are converted from milliseconds to a human-readable format (minutes and seconds) for display.

IMPLEMENTING INTERACTIVE CHAPTER PLAYBACK

To enhance user experience, each chapter summary is presented within an expander. A key feature is the clickable button associated with each chapter, which, when pressed, jumps the audio player to the specific start time of that chapter. This interactivity is achieved using Streamlit's session state and callback functions. A callback function updates a `start_point` session state variable with the chapter's start time, dynamically controlling the audio player's playback position.

ADVANCED FEATURES AND USER INTERFACE ENHANCEMENTS

The application leverages Streamlit's expanders to organize content, preventing the interface from becoming cluttered, especially with multiple chapter summaries. Markdown is used for presenting lists of themes and chapter summaries. The use of pandas DataFrames simplifies data manipulation and display, particularly for chapter information like start/end times. This structured approach makes the summarized meeting data accessible and easy to navigate for the user.

DYNAMIC AUDIO PLAYER CONTROL WITH SESSION STATE

A significant aspect of the application's functionality is its dynamic audio player. By utilizing Streamlit's session state, specifically a variable like `start_point`, the application can track the desired playback position. Callback functions are triggered when a user interacts with a chapter's 'play' button, updating this `start_point` state. The audio widget is then configured to respect this state, allowing it to jump to the selected chapter's timestamp, thus enabling targeted listening.

CONCLUSION AND FURTHER DEVELOPMENT POTENTIAL

The tutorial concludes by showcasing the fully functional application, which successfully processes audio files into structured summaries and interactive chapter playback. The video encourages viewers to like, subscribe, and comment with questions. It also reiterates the availability of a free AssemblyAI API token through a provided link for viewers to experiment with the technology and replicate the tutorial's success.

Mentioned in This Episode

●Products

●Software & Apps

●People Referenced

Building an Auto-Meeting Summarizer with Python

Practical takeaways from this episode

Do This

Use Streamlit for the UI to upload audio and display results.

Utilize Assembly AI's API for speech-to-text, sentiment analysis, and auto-chaptering.

Integrate Python libraries like 'requests' for API calls and 'pandas' for data manipulation.

Implement polling mechanisms to check transcription status.

Display key themes and chapter summaries in user-friendly expanders.

Use session states and callback functions for interactive elements like clickable chapter timestamps.

Convert timestamps from milliseconds to seconds/minutes for better readability.

Organize code into separate files for clarity (e.g., API interaction, configuration).

Avoid This

Do not hardcode API keys; store them securely (e.g., in a config file).

Avoid displaying raw JSON data; format it for user readability.

Do not let the application freeze while waiting for transcription; use polling.

Do not clutter the main application file with too many helper functions.

Meeting Chapter Summaries

Data extracted from this episode

Gist	Summary	Start Time (MM:SS)	End Time (MM:SS)
Trip to the Past	The cast of the Lord of the Rings is on a reunion call. Joan and Josh are trying to reunite some of the cast behind the iconic trilogy. Orlando Bloom is now on the call but he is not a fan of the series.	00:00	01:02
Lord of the Rings Reunion	Elijah Wood, Sean Astin, Dominic Monaghan, etc. reunite. Elijah Wood says an iconic line. Peter Jackson apparently also joins.	01:02	02:01
Theft from Set	If you have a prop from the Lord of the Rings, pull it out now. The speaker is impressed that everybody got something. This is called theft.	02:01	02:19
Score and Themes	The fellowship theme and the Shire theme are the other half of the film. Howard Shore is the composer of the iconic ring theme. Peter and Philip first recorded the score.	02:19	03:34

Common Questions

You can build an application using Python and a service like Assembly AI. This involves uploading the meeting's audio file, using the API for transcription and analysis, and then displaying the extracted themes and chapter summaries.