On November 2, OpenAI introduced LifeVal, an evaluation of AI performance on everyday tasks. To select areas of interest, the company drew on time-use research and a survey of 3,995 people. Official publication.
What follows is our editorial analysis of how readers can assess such tests and their own experience with an assistant. It is not an independent evaluation of LifeVal or a report of hands-on model testing.
Define the finish line first
Imagine asking an assistant to find a suitable after-school activity. A neat comparison table does not necessarily finish the job. It could contain classes for the wrong age group, outdated schedules or locations that are impossible to reach after school.
A useful test should therefore explain what success means: a list of links, a verified shortlist or a confirmed registration.
This is an editorial example, not a task taken from the published evaluation. It illustrates the difference between the quality of the writing and the quality of the work. Readers need an outcome they can use without repeating the whole search themselves.
Four questions for any benchmark
First, who decides whether the task succeeded? A test designer, an independent reviewer and the actual user can reach different conclusions. Everyday work depends heavily on individual constraints. A route involving a transfer can be technically correct while unsuitable for a family travelling with a stroller.
Second, what conditions does the system receive? Browser access, current information and working tools influence what is possible. When comparing results, check the testing environment as well as the model name.
Third, how are mistakes counted? A broken link and an incorrectly submitted application have very different consequences. A single percentage can hide that distinction. We would want an assistant evaluation to separate search accuracy, compliance with constraints, correct actions and the ability to stop when necessary.
Fourth, what work remains for the person? If every item needs to be checked again, the time saving may disappear. A report that makes the outcome verifiable is more useful than a confident message saying the job is done.
Run a small test of your own
Start with a reversible task and a clear result, such as comparing several public schedules against criteria you define in advance. Write down the non-negotiable requirements before starting. Then compare the answer with the original pages and count both the assistant's time and the time you spend checking it.
Keep track of well-timed questions, too. Clarifying the budget or date can be more valuable than answering a misunderstood request quickly. Acknowledging missing information should not automatically count against an assistant.
An initial test does not require a passport, banking details or access to every conversation you have had. A narrower task makes it easier to understand why something worked or failed. Moving on to applications, payments and other consequential actions requires separate user oversight.
What to look for next
Our interest in LifeVal comes down to how clearly a reported score can be connected to a completed real-world task. Reproducibility, the testing conditions and analysis of failures all matter. Developer-reported results should be read as results of a particular evaluation, rather than a promise of equal reliability in every situation.
For readers, the practical question remains how much work the assistant removed and how much checking it created. That balance makes a discussion of AI more useful than another contest between percentages.
Comments