Humanoid Robot meets ChatGPT – by Shubham Saboo

Today’s top AI Highlights:

1. Figure + OpenAI > The most “humanoid” robot

2. DeepMind’s generalist AI agent for 3D virtual environments

3. The most important of the new EU AI Act

4. OpenAI’s Sora will be publicly available by 2024 end

5. First-ever go-to-market AI Platform: the all-new Copy AI

& so much more!

Read time: 3 mins

Figure has been sharing impressive developments of its humanoid robot Figure 01. The company on February 29, 2024, announced a strategic partnership with OpenAI to leverage their generative AI technology for training Figure 01, particularly in processing and reasoning from language. And in just 15 days, they have released a demo which we feel is by far the most impressive one we’ve seen.

You’ll see Figure 01 communicating with the human instructor and carrying out tasks. The robot describes its visual experience, plans future actions, reflects on its memory, and explains why it did what it did verbally. The surprising part is how “human” the robot feels with the sheer dexterity and language, even the human-like stuttering!

The company claims there are no teleoperations, filming speed is 1.0X, all captured in real-time, and the footage is unedited. The cameras in Figure 01 feed into OpenAI’s vision-language model that allows the robot to process visual information at 10hz and translate it into 24 degrees of freedom actions at an impressive 200hz.

While the robot takes a little time to respond, certainly we are not far from witnessing these robots being deployed to manufacturing facilities, caregiving, personal assistance, education, and everywhere else!

AI in video games has always been about mastering specific titles, from the early days of playing Atari to the sophisticated system of AlphaStar dominating StarCraft II. But here is an AI wingman that does what you tell it to do. Google DeepMind’s SIMAthe Scalable Instructable Multiworld Agent, is capable of navigating and performing tasks in multiple 3D video game environments based on natural language instructions. You could tell SIMA to do tasks like navigating to a specific location, gathering resources, or crafting items and it would carry out those tasks.

Key Highlights:

1. SIMA was trained and tested on nine different video games and four specially designed research environments. SIMA developed a broad understanding of 3D virtual spaces involving on-screen text, complex landscapes, interacting with objects, and applying the principles of the physical world within these digital realms.

2. This training wasn’t just about exposing SIMA to various game mechanics. It also involved videos with one player watching and instructing the other to make it understand how language ties in with game-play behavior.

3. Without needing access to game source code or APIs, SIMA uses just screen images and human instructions to control game characters. It performs keyboard and mouse actions, mimicking how humans play any game.

4. One of SIMA’s most impressive features is its ability to generalize across different games. SIMA demonstrated near-equivalent performance in games it hadn’t been specifically trained on, in comparison to other AI agents that specialize in that specific environment.

After being proposed in 2021, hotly debated, and reported lobbying by big AI companies, the EU Parliament has finally approved the landmark AI Act with a majority. The Act employs a risk-based framework, categorizing AI systems into four levels of risk: unacceptable, high, limited, and minimal, where high-risk systems will need to adhere to the strictest requirements like risk mitigation, high-quality datasets, and human oversight in areas like employment, credit, and healthcare.

Key Highlights:

1. Companies are now required to notify people when interacting with AI systems, including chatbots and biometric categorization, and mandate the labeling of deepfakes.

2. The Act prohibits certain uses of AI, like real-time biometric identification in public spaces, emotion recognition systems in law enforcement and educational institutions, and indiscriminate scraping of biometric data.

3. A new European AI Office will be established to coordinate compliance, implementation, and enforcement of the Act.

4. The AI Act does not apply to AI systems developed exclusively for military and defense uses, focusing instead on civilian applications.

5. Companies face fines ranging from 1.5% to 7% of global sales turnover for noncompliance, depending on the offense’s severity and the company’s size.

OpenAI has almost been “teasing” the AI community with the demo videos of its text-to-video model Sora. We all had been waiting to know when the model would be publicly available for use, Mira Murati, CTO of OpenAI, has announced that it’ll be publicly available later this year, possibly within a few months.

Sora is currently available only to visual artists, designers, and filmmakers. OpenAI is further planning to incorporate audio into Sora to bring the videos to life, exactly like Pika’s Sound Effects. They will also enhance the model to allow users to edit the videos generated by Sora.

While AI companies have been sued by a lot of artists for copyright infringement, Murati did not reveal the training data details except that it was publicly available or licensed data and a partnership with Shutterstock to train the model.

1. Copy AI: The first GTM AI Platform designed to power sales and marketing processes through automation, including content creation, prospecting, CRM enrichment, and much more. It features a user-friendly interface, requires no special knowledge in prompt engineering, offers high-quality data outputs, integrates seamlessly with 2,000+ tools, and maintains a zero-retention data policy for security.

2. Auto Label by Roboflow: Automate labeling of thousands of images for computer vision applications. This platform enables users to test prompts, batch label images, and review annotations to enhance model training efficiency and accuracy.

3. Airfront: AI-first email platform that understands the context behind each email and enables smart workflows such as responding to customers and syncing data. It’s quick to set up and automates routine work like managing inbound, writing follow-ups, tracking schedule, running micro campaigns, and more.

4. Fructose: A Python package that provides a dependable, strongly typed interface for calling LLMs through a simple decorator. By using the @ai decorator on a function with type annotations, Fructose allows for seamless integration of AI-generated content into Python applications.

😍 Enjoying so far, TWEET NOW to share with your friends!

1. AI will probably be smarter than any single human next year. By 2029, AI is probably smarter than all humans combined. ~
   Elon Musk

2. It’s amazing how quickly the “current thing” changes in the AI space!
   Hype cycles last no more than a day!
   Yesterday everyone was giddy about Grok being open-sourced and today we are actually entertaining the possibility of software engineering being automated in 2024! ~
   Bindu Reddy

3. A big corporate misconception of Generative AI: Pre-LLM AI was training models on your data to find hidden insight. Firms are trying to do that with LLMs but they don’t really work that way, they aren’t analysis tools & your data matters less, they are pre-trained on the internet ~
   Ethan Mollick

That’s all for today!

See you tomorrow with more such AI-filled content. Don’t forget to subscribe and give your feedback below 👇

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!

Share Unwind AI