النماذج المحلية تتفوق على نماذج السحابة في اختبار التسوق على أمازون

وجدت الدراسة أن نموذج لغوي محلي أصغر (~3 مليارات معلم) حقق معدل نجاح مثالي 7/7 في إكمال عملية تسوق معقدة على أمازون، متفوقًا على نموذج سحابي أكبر نجح مرة واحدة فقط. كما استخدم النموذج المحلي رموزًا أقل وتحمل تكلفة صفرية إضافية، مما يظهر أن التصميم المعماري يمكن أن يتجاوز القوة الحسابية الخام.

استخدم النظام معمارية ثلاثية الأجزاء: قام بتقليم DOM لتقليل التعقيد، وفصل التفكير عن التنفيذ بين نموذجين متخصصين، ونفذ حلقة تحقق مع تأكيدات لكل خطوة. هذا ضمن أن الوكيل لا يمكنه المضي قدمًا إلا بعد إثبات نجاح كل إجراء، مما أزال التخمين.

Key Facts: 1. A local ~3B parameter LLM successfully completed a full Amazon shopping flow with a 7/7 success rate using only structural page data. 2. The local model stack operated with zero incremental cost and required no vision capabilities, contrasting with expensive cloud API calls. 3. The system reduced input complexity by pruning approximately 95% of DOM nodes, creating a compact semantic snapshot for the model. 4. The local model used 11,114 tokens compared to the cloud model's 19,956 tokens, demonstrating greater efficiency in token usage. 5. The verification layer implemented Jest-style assertions after every action, ensuring the agent could only proceed after proving state changes. 6. The experiment concluded that constraining the state space and making success explicit through verification is more effective than scaling model size. FAQ: Q1: What was the main finding of the Amazon shopping automation test? A1: The study found that a smaller, local language model (~3B parameters) achieved a perfect 7/7 success rate in completing a complex Amazon shopping flow, outperforming a larger cloud model that only succeeded once. The local model also used fewer tokens and incurred zero incremental cost, demonstrating that architectural design can trump raw computational power. Q2: How did the local model achieve such high reliability? A2: The system used a three-part architecture: it pruned the DOM to reduce complexity, split reasoning from acting between two specialized models, and implemented a verification loop with per-step assertions. This ensured the agent could only proceed after proving each action was successful, eliminating guesswork. Q3: What are the implications for AI agent development? A3: The results suggest that for reliable automation, developers should focus on constraining the problem space and implementing rigorous verification checks rather than simply using larger models. This approach reduces costs, improves success rates, and makes agent behavior more predictable and trustworthy. Q4: Was there a trade-off in performance? A4: Yes, the local model stack was significantly slower, taking about 405 seconds compared to the cloud model's 60 seconds. However, the local model's perfect success rate and zero cost made it more practical for scenarios where reliability is prioritized over speed.