JD.com Open-Sources 24B-Parameter Image Model JoyAI-Image-Edit, Focusing on Spatial Intelligence Editing Capabilities
JD Explore Academy announced in early April 2026 the official open-sourcing of its self-developed 24-billion-parameter multimodal image editing model, JoyAI-Image-Edit. The model’s core feature is the integration of “spatial intelligence” into its underlying architecture, aiming to address the limitations of existing generative AI in handling 3D spatial relationships and geometric consistency. The model’s code, inference pipeline, and weights have been released on platforms like GitHub and ModelScope under the Apache 2.0 open-source license, allowing for commercial use and secondary development by developers and enterprises.
A Unified Architecture Fusing Understanding and Generation
JoyAI-Image-Edit’s technical foundation lies in its innovative unified architecture, which contains a total of 24 billion parameters. Specifically, it couples an 8-billion-parameter multimodal language model (MLLM) with a 16-billion-parameter Multimodal Diffusion Transformer (MMDiT), forming a shared interface.
The core idea behind this design is to break the traditional separation of “understanding” and “generation” modules in image models, establishing a collaborative closed loop. In this framework, enhanced spatial understanding provides more precise control for generation and editing tasks. In turn, the output of generative operations (like viewpoint changes) provides new data and evidence for the model’s spatial reasoning. To achieve this, the research team built a scalable data pipeline, including the OpenSpatial dataset for spatial understanding, long-text rendering data, and the SpatialEdit dataset for editing tasks, and trained the model using a multi-stage optimization strategy.
From a practical technical standpoint, this closed-loop mechanism is key to achieving advanced spatial intelligence. It enables the model to be more than a simple pixel manipulation tool, allowing it to begin simulating the human cognitive process of the 3D world—continuously refining its representation of the spatial environment through observation (understanding) and interaction (generation/editing).
3D Controllable Editing with Spatial Awareness
Based on its unique architecture, JoyAI-Image-Edit demonstrates highly controllable editing capabilities in the spatial dimension, primarily through three prompt paradigms:
Camera Control: Users can issue natural language commands, such as “pan the camera 30 degrees to the right, tilt down 15 degrees, and zoom in,” and the model can generate a new image from the requested viewpoint. During this process, the scene’s geometric structure, object occlusion relationships, and lighting effects remain highly consistent and physically realistic.
Object Rotation & Spatial Traversal: The model supports rotating specific objects within a scene or generating a continuous sequence of multi-view images. This capability is akin to “filming” by moving a camera within a virtual 3D scene, ensuring logical coherence between frames.
Object Spatial Relation Manipulation: Users can precisely move, rotate, or scale an object in the image. When performing these operations, the model intelligently handles occlusion, lighting changes, and perspective, avoiding common issues in traditional editing methods like distortion, misalignment, or imbalanced proportions.
In addition to its outstanding spatial editing capabilities, JoyAI-Image-Edit also supports about 15 general editing tasks, including object replacement, deletion, style transfer, and long-text rendering, showcasing its versatility as an integrated image editing tool.
Focusing on Two Major Application Scenarios: E-commerce and Embodied AI
JoyAI-Image-Edit’s technical features give it significant application potential in two areas closely related to JD.com’s core business: e-commerce and embodied AI.
In e-commerce, multi-angle and multi-scene product displays are a key marketing demand. Traditionally, this requires substantial costs for photography and post-production. Using JoyAI-Image-Edit’s spatial editing capabilities, merchants can automatically generate dozens of marketing assets with different viewpoints, backgrounds, and compositions based on one or a few product images. This maintains the consistency of the product’s shape and details, thereby significantly reducing content production costs.
In the field of embodied AI, a core challenge for robots is to accurately understand and interact with the physical world, which relies heavily on precise perception of 3D space. JoyAI-Image-Edit can serve as a foundational model for a robot’s vision system, providing it with powerful spatial understanding capabilities. More importantly, it can efficiently generate large-scale, high-quality synthetic training data—for instance, by simulating various robot actions through commands. This helps alleviate the long-standing bottleneck of insufficient real-world data collection in the robotics industry.
Open-Source Strategy and JD.com’s Industrial AI Layout
JD.com’s choice to open-source the model under the business-friendly Apache 2.0 license reflects its strategy to promote the development of the technology ecosystem. By releasing a model with advanced spatial consistency capabilities to the community, it aims to lower the technical barrier, providing a solid foundational platform for e-commerce professionals, designers, robotics developers, and academic researchers.
This open-source initiative also reveals JD.com’s strategic positioning in the AI field: it is not just about chasing parameter scale or benchmark rankings, but about deeply integrating AI technology into its advantageous industrial scenarios. From the previously open-sourced JoyAI-LLM Flash foundation model and the digital avatar JoyStreamer for live streaming, to the current JoyAI-Image-Edit, JD.com’s AI projects have consistently revolved around real-economy sectors like retail, logistics, industry, health, and embodied AI.
This approach, guided by scenario-based implementation and creating practical value, indicates that the ultimate AI competition will return to the ability to solve specific problems. By rooting AI technology in its most familiar industrial soil, JD.com aims to build a complete closed loop from technology R&D to scene application.
