OpenAI O3 Mini vs DeepSeek para uso agentivo

A partir del 31 de enero, OpenAI lanzó su nueva generación de modelos de razonamiento con el modelo O3 Mini. En general, parece que el modelo OpenAI O3 Mini es 15 veces más barato y tiene aproximadamente la misma precisión que el modelo O1, la generación anterior de modelos de razonamiento de OpenAI. El nuevo modelo, OpenAI O3 Mini, ya está disponible en FlowHunt a menos de un día de su lanzamiento por parte de OpenAI. En este blog, profundizaremos en las capacidades del modelo y lo compararemos con Deepseek.

Métricas de evaluación para OpenAI O3 Mini vs DeepSeek

En esta evaluación, utilicé un flujo de trabajo sencillo pero eficaz que implica entrada de chat, salida de chat y un generador de IA para ofrecer una explicación clara y concisa de la línea principal de la apertura Ruy López en ajedrez. Al comparar el rendimiento de DeepSeek con O3 Mini, pude evaluar cómo manejaba cada sistema la tarea de describir los movimientos clave—como 1. e4 e5, 2. Nf3 Nc6, 3. Bb5 a6, 4. Ba4 Nf6 y 5. O-O Be7—y sus desarrollos posteriores. Este análisis comparativo no solo demuestra las capacidades de estas herramientas de IA para procesar y transmitir estrategias de ajedrez complejas, sino que también destaca el valor de un flujo conversacional simple para lograr resultados legibles y evaluaciones prácticas.

Este es el flujo:

Resultados de DeepSeek

Le pregunté al modelo lo siguiente:
“Give me first 10 moves of Ruy Lopez.”
El resultado es el siguiente. Este es el proceso de pensamiento de DeepSeek:

o3 mini vs deepseek Restarting session Please wait give me first 10 moves of ruy lopez
Okay, so I’m trying to figure out the first 10 moves of the Ruy Lopez opening in chess. I remember that the Ruy Lopez is one of the oldest and most popular openings, starting with the moves 1.e4 e5. Then, White plays 2.Nf3, developing the knight and preparing to control the center. Next, 2…Nc6, where Black develops their knight and supports the e5 pawn. Now, the Ruy Lopez proper starts with 3.Bb5, which attacks the knight on c6. I think this is the key move that defines the opening. Then, Black usually responds with 3…a6 to challenge the bishop. After that, White plays 4.Ba4, retreating the bishop to a safe square. So far, I have six moves: 1.e4 e5 2.Nf3 Nc6 3.Bb5 a6 4.Ba4. Now, Black’s next move is often 4…Nf6, developing the knight and attacking the pawn on e4. Then, White castles kingside with 5.0-0 to get the king to safety and bring the rook into the game. That’s eight moves. Next, Black might play 5…b5 to challenge White’s bishop on a4 and gain space on the queenside. Then, White responds with 6.c3, supporting the d4 push and preparing to develop other pieces. So, the tenth move would be 6…d6, solidifying the e5 pawn and preparing to develop the dark-squared bishop. Wait, but I’m not entirely sure if Black always plays 5…b5. Sometimes, they might play 5…Be7 instead, but I think 5…b5 is more common in the Closed Ruy Lopez. Also, after 6.c3, Black can choose between 6…d6 or 6…b5 again, but 6…d6 seems more standard to support e5. I should double-check the order to make sure I didn’t mix up any moves. Let me go through it again: e4 e5 Nf3 Nc6 Bb5 a6 Ba4 Nf6 0-0 b5 c3 d6 Yes, that seems correct. So the first ten moves are as I listed. I think that’s a solid foundation, but I’m a bit unsure if there are alternative lines Black might take earlier, like after 3.Bb5, maybe 3…g6 instead of 3…a6, but I think 3…a6 is the main line leading to the Morphy Defense. So, I’ll stick with that for the first ten moves. Foundation for both White and Black, with White aiming for a strong center and piece development, while Black challenges White’s bishop and prepares for further development.

Resultado final:

The first ten moves of the Ruy Lopez opening, following the Morphy Defense, are as follows:
e4 e5 Nf3 Nc6 Bb5 a6 Ba4 Nf6 0-0 b5 c3 d6
This sequence establishes a solid foundation for both White and Black, with White aiming for a strong center and piece development, while Black challenges White’s bishop and prepares for further development.

Resultados de OpenAI O3 Mini

O3 gana. La respuesta de O3 Mini es más comprensible y correcta.

The Ruy Lopez (or Spanish Game) is a rich opening with many variations. One of the most common ways to reach a classical Closed Ruy Lopez is with the following moves:
e4 e5 Nf3 Nc6 Bb5 a6 Ba4 Nf6 O-O Be7 Re1 b5 Bb3 d6 c3 O-O h3 Nb8 d4

Here’s a brief explanation of the ideas behind these moves:

• 1. e4 e5 sets up the classical battle for the center.
• 2. Nf3 develops a knight and attacks the e5 pawn; Black defends with Nc6.
• 3. Bb5 pins the knight on c6 (which is defending e5) and increases central pressure; Black challenges with a6.
• 4. Ba4 retreats while still keeping the pin; Black develops another piece with Nf6.
• 5. O-O secures White’s king; Black plays Be7 to prepare castling.
• 6. Re1 reinforces the central e4 pawn; Black’s b5 kicks the bishop and gains space on the queenside.
• 7. Bb3 preserves the bishop’s diagonal; Black shores up the center with d6.
• 8. c3 prepares to support a later d4 push while also preparing for a possible retreat of the bishop if needed, and Black castles with O-O.
• 9. h3 prevents any piece (like a knight or bishop) from coming to g4, where it could harass White’s center or pieces; Black retreats the knight with Nb8 to re-route it.
• 10. d4 is White’s break in the center, intending to challenge Black’s pawn structure.

Keep in mind this is just one of many valid move orders in the Ruy Lopez. Variations such as the Open Ruy Lopez or other lines in the Closed Ruy Lopez might differ after move 4 or later. But the sequence above is a well‐known path that many players follow in classical chess theory.

Evaluando DeepSeek y OpenAI O3 Mini en comportamiento agentivo

Hemos evaluado los modelos de razonamiento en una tarea que requiere de 2-3 herramientas para agentes de IA. Esta es la tarea:

Dado el nombre de la empresa, busca y encuentra la página de precios del producto y sugiere mejoras.
Además, toma una captura de pantalla de la página de precios.
Luego envíame un correo electrónico con los resultados.

La tarea requiere conectarse a Google para buscar la página de precios, rastrear la página y enviar un reporte detallado por correo electrónico.

La tarea fue intencionalmente poco detallada para evaluar las capacidades de razonamiento de los modelos mencionados.

Usando DeepSeek como agente de IA

Deepseek no pudo enviar el correo electrónico. En general, nuestra experiencia revela que Deepseek no es tan preciso en la llamada de herramientas en comparación con OpenAI GPT4o u OpenAI O3 Mini. Este es el informe generado por DeepSeek, analizando la página de precios de FlowHunt. Este es el proceso de pensamiento de DeepSeek:

1. Llamada a la herramienta de Google:

2. Extracción de URL:

El razonamiento:

Subject: FlowHunt Pricing Page Review and Suggestions

Dear [Your Name],

I hope this message finds you well. Below is a summary of the FlowHunt pricing page, along with some suggestions for enhancements.

Pricing Overview:

FlowHunt offers four pricing tiers:

Starter Plan (€10/month):

• 10 Credits
• 5 Workflows per Workspace
• 1 Workspace
• 1 Team Member per Workspace
• 5 Chatbots per Workspace
• 2 Websites per Workspace
• 5 Documents per Workspace
• 50 FAQs per Workspace
• Access to All Latest OpenAI LLMs, Including Custom Fine-Tuned Models
• All Integrations
• Email Support

Pro Plan (€50/month):

• 55 Credits (Includes 10% Discount on Credits)
• 20 Workflows per Workspace
• 5 Workspaces
• 10 Team Members per Workspace
• 20 Chatbots per Workspace
• 10 Websites per Workspace
• 50 Documents per Workspace
• 100 FAQs per Workspace
• Access to All Latest OpenAI LLMs, Including Custom Fine-Tuned Models
• Pro Support
• API Key Access
• Basic Analytics (Coming Soon)
• All Integrations

Premium Plan (€500/month):

• 600 Credits (Includes 20% Discount on Credits)
• 50 Workflows per Workspace
• 10 Workspaces
• 100 Team Members per Workspace
• 50 Chatbots per Workspace
• 30 Websites per Workspace
• 100 Documents per Workspace
• 200 FAQs per Workspace
• Access to All Latest OpenAI LLMs, Including Custom Fine-Tuned Models
• Priority Support
• API Key Access
• Advanced Analytics (Coming Soon)
• Initial Workflow Consultation
• Initial Workflow Training Support
• All Integrations

Enterprise Plan (Custom Pricing starting from €1500/month):

• Contracted Tokens (Bring Your Own Key – BYOK)
• Unlimited Workflows
• Unlimited Workspaces
• Unlimited Users
• Unlimited Chatbots
• Unlimited Schedules
• Unlimited Documents
• Unlimited Questions and Answers
• Unlimited Chat History
• Access to All Latest OpenAI LLMs, Including Custom Fine-Tuned Models
• Dedicated Priority Support
• API Key Access
• Advanced Analytics (Coming Soon)
• Customizable Workflow Modules
• Optional Training Support
• Advanced Security and on-premises Deployment

Suggested Enhancements:

• Feature Comparison Table: Create a side-by-side comparison table highlighting the features of each plan. This would make it easier for users to quickly identify the differences between the plans.
• Customer Testimonials/Case Studies: Adding testimonials or case studies from existing customers can build trust and credibility.
• Pricing Calculator: Implement an interactive pricing calculator that allows users to estimate their costs based on their specific needs, especially for the Enterprise plan.
• Free Trial Highlight: Make the free trial option more prominent on the page to encourage users to try the service before committing to a paid plan.
• Detailed Documentation/Setup Guides: Provide links to detailed documentation or setup guides to help users understand how to make the most of the features included in each plan.

Usando OpenAI O3 Mini como agente de IA

Por otro lado, con esta escasa descripción, el modelo OpenAI O3 Mini fue capaz de descifrar el proceso exacto:

1. Búsqueda en Google

2. Extracción de URL

3. Envío de correo electrónico

Este es el correo que recibí:

Hello,

Below are the results of the review of FlowHunt’s pricing page and some suggested enhancements:

Pricing Page URL: https://www.flowhunt.io/pricing/

Screenshot: A screenshot of the pricing page was taken. (Please refer to the attached screenshot in your system.)

Summary of the Pricing Page:

The pricing page clearly displays four plans: Starter (€10/month), Pro (€50/month), Premium (€500/month), and Enterprise (from €1500/month). Each plan lists the credits, workflow and team limits, core features, and additional support options. The page includes both an overview and a detailed comparison using a table format, which helps potential customers compare the features across plans.

Suggested Enhancements:

• Interactive Pricing Calculator: Consider adding an interactive pricing calculator that allows users to adjust parameters (e.g., number of workflows, team members) to see a dynamic estimate of their monthly cost.
• Visual Enhancements and UX Improvements: Introduce more engaging visuals (e.g., icons or infographics) to further differentiate the features of each plan. Add subtle animations when hovering over different plan details to improve user interactivity.
• Currency Converter: Since pricing is in Euros, include a currency converter option to cater to international users for a better localized experience.
• Clearer Call-to-Action (CTA): Enhance the CTA buttons by making them more prominent and possibly adding a secondary CTA (e.g., “Learn More” about plan benefits) for users who need more clarity before signing up.
• Enhanced FAQ Section: Expand the FAQ section with more detailed answers regarding the credits system, potential overages, and common customer scenarios. Consider using collapsible sections to keep the page clean while providing additional information when needed.
• Social Proof and Testimonials: Incorporate customer testimonials or success stories near the pricing section to build credibility.
• Mobile Optimization: Ensure that the pricing table and all interactive elements render optimally on mobile devices.
• Chat Support Integration: Add an integrated live chat or chatbot support option on the pricing page to instantly address any queries related to pricing or plan details.

By implementing these enhancements, FlowHunt can provide a more engaging and user-friendly experience, potentially increasing conversion rates.

Best regards,
[Your Name]

(P.S. This email includes the results and suggestions for FlowHunt’s pricing page as well as information about the captured screenshot.)