|
China’s New Open-Source AI Just Crushed DeepSeek! (7/27/2025) |
This video describes a new large
language model (LLM) called M1 released by the Chinese company
Miniax. Key points about M1 from the video and transcript are:
M1 is a
groundbreaking open-source LLM with an extraordinarily large
processing capacity, capable of handling 1 million input tokens and
generating up to 80,000 token responses. This is far beyond previous
models like GPT-4 (which handles about 125k tokens input), Claude 4
Opus, and DeepSeek R1, which max out around 128k tokens.
The model
architecture is innovative: It contains 456 billion parameters in
total but uses a Mixture of Experts framework that activates only
about 46 billion parameters per operation by loading 32 specialized
subnetworks depending on the tokens processed. This reduces
computational load dramatically.
M1 uses a
proprietary "Lightning Attention" mechanism instead of traditional
quadratic attention, giving it linear scaling with sequence length
and enabling efficient handling of long texts without prohibitive
resource use.
Training was completed in about three
weeks using 512 Nvidia H800 GPUs at a rental cost of approximately
$535,000, significantly cheaper than DeepSeek's reported $5-6
million and far below GPT-4's estimated over $100 million training
cost due to architectural and training optimization innovations.
It is fully open
source with permissive licensing allowing complete on-premises
deployment and no usage or subscription fees, aiming to democratize
advanced AI capabilities worldwide.
M1's strengths
include extended context understanding (handling millions of
tokens), competitive reasoning and coding performance, and flexible
task adaptability. It slightly trails DeepSeek on some benchmarks
but exceeds other open models.
The project
recommends using VLLM as a backend for serving the model for
efficient memory pooling and expert model handling, with other
standard transformer libraries also supported.
The release
represents a major leap for open-source AI in capability,
efficiency, and accessibility, potentially shifting global AI power
balance.
Regarding smaller quantized versions, the
video transcript and description do not explicitly mention quantized
or smaller lightweight versions of the M1 model being released yet,
though given the scale of the model (hundreds of billions of
parameters globally, 46 billion active), future quantized variants
would be relevant for wider practical deployment.
In summary:
Name: M1 language
model by Miniax
Size: 456 billion
parameters total, 46 billion active per token operation
Unique features: 1
million token input, Lightning Attention (linear scaling), Mixture
of Experts, very low computational training cost
Open source: Yes,
fully open source with permissive licensing, free with no usage fees
Quantized smaller
versions: Not explicitly mentioned yet
Deployment:
Supports VLLM backend, on-premises use possible
This model marks a
significant advancement in open LLMs enabling unprecedented
extended-context language understanding and cost-effective training
compared to previous giants like DeepSeek and GPT-4. It is already
being integrated across Asia and poses a challenge to Western
closed-model dominance in AI. |
Back
to Mike Beaver's Hypnotherapy Home Page
|
|
If you are the original creator of material featured on this website and want
it removed, please contact the webmaster.
|