Managing prompts and models often requires tedious code tweaks and constant software updates. Now, imagine if a single dashboard streamlined this entire process. With no additional code changes needed, your workflow becomes effortlessly efficient. Moreover, real-time monitoring is typically a hassle. Imagine capturing every LLM request and interaction, offering a comprehensive view that enables your team to seamlessly evaluate completion data. Throughout Q1 2024, I led a redesign to enhance these features and address additional opportunities. Here are a few of the transformative changes we shipped.
We re-themed the Tests feature, improving its usability and aesthetics. I also refactored the feature to allow for comparisons between test runs. In the process, I enhanced the test run outputs to display the number of evaluations and enabled prompt comparison, ensuring users can make informed evaluations efficiently.
We simplified the creation and iteration of prompt templates. This enhancement empowers users to craft multiple versions swiftly while deploying to the environment of their choosing. Additionally, the feature now supports the inclusion of test variables, facilitating more dynamic and tailored testing scenarios.
Redesigned the data visualizations for session prompt evaluations, focusing on clearer, more actionable insights. In a similar vein, session details were overhauled to allow for individual evaluations and prompt editing, enhancing user control. Furthermore, I introduced hydrated variables to ensure that variables are automatically updated with their current values, as opposed to variable names.
We iterated on Sessions View table filtering to be more intuitive, now contained within a dialog - as opposed to many filters. This update also introduces the ability to save views with applied filters, effectively creating a new session group, simplifying navigation and enhancing user productivity.
We sought to enhance visibility and enhance functionality of Datasets, now including data visualizations for labelled examples and evaluation criteria. Additionally, the dataset detail view is now editable, with the added capability of manually creating a dataset, offering users more flexibility and control over their data management.
© Yordani Awono 2024 , v.01