Medusa vs Magento: Performance comparison

By Grzegorz Tomaka

Featured image

Medusa is 6.5x faster than Magento! Find out in the article how we checked it.

Magento is an old player in the eCommerce field, known for its comprehensive but complex system. Recently, it's shifted towards a headless approach, trying to keep up with the modern digital market. But does this shift put it on par with newer, API-first platforms like Medusa, designed from the ground up to be flexible and efficient?

This comparison is about seeing if Magento, with its new headless approach, can match the performance of platforms built to be headless from day one.

What is Medusa?

medusa-strona

Medusa is an open-source, headless commerce platform built with modern JavaScript. It's designed to suit any eCommerce need, including complex setups like multi-vendor marketplaces. Its API-first approach and decoupled architecture enable rapid development and scalability.

What is Magento (Adobe Commerce)?

magento-strona

Magento, now Adobe Commerce, is a leading open-source e-commerce platform known for its extensive feature set. It supports a headless architecture, allowing businesses to create custom front-end experiences while leveraging Magento's powerful e-commerce capabilities on the backend.

What We Compare

In our comparison of Medusa and Magento, we're diving deep into API performance, focusing on how efficiently each platform handles data requests. API performance is crucial because it directly impacts the speed and responsiveness of an eCommerce site, affecting everything from user experience to SEO rankings.

During API performance testing, we will measure average and 95th percentile load time based on http_req_duration.

http_req_duration

To compare Medusa and Magento's API performance, we focus on the metric http_req_duration This metric represents the total time it takes for an HTTP request to complete, from the initial request to the final byte of the response. It encompasses all aspects of the request cycle, making it a comprehensive measure of the efficiency of your e-commerce platform's backend.

The Importance of Average Loading Time and the 95th Percentile

  • Average Loading Time: This metric gives us a general idea of how long it typically takes for an API call to load across all tests. It's helpful in establishing a baseline performance but can sometimes hide issues like occasional delays or outliers.
  • 95th Percentile: Instead of just looking at averages, we focus on the 95th percentile to better understand performance under load. This metric helps us see how the platform performs under stress, excluding the most extreme outliers. It's crucial for e-commerce because it reflects the experience of nearly all users.

Why is Loading Speed Important?

Loading speed is crucial because even minor delays can drastically lower conversion rates and sales. Fast-loading sites meet user expectations, improve SEO rankings, and are more likely to keep users engaged. In short, a quicker website directly translates to a better user experience and increased revenue.

To learn more, check out our other article - "How to Increase Sales in the Online Store?"

Test Cases: Why Focus on Product Fetching?

We've focused on fetching product data as a primary test case for our comparison. This decision is based on the premise that viewing product information is often the first step in a customer's journey to purchase. Efficiently loading product details keeps potential buyers engaged and reduces bounce rates.

How We Conduct the Test

Here's how we're conducting our performance test:

Tool k6 for Load Testing

For our testing, we're using k6, a powerful open-source load-testing tool. k6 is built to simulate real-world traffic on web applications, providing insights into how systems perform under various conditions. k6 allows us to simulate traffic and examine API performance.

To ensure our test results focus purely on server response times and are not influenced by network latency variations, we've set up our k6 testing environment directly on the server.

Testing Environment: Uniform Server Specifications

We're running both Medusa and Magento on identical server setups to eliminate hardware as a variable. Each platform is hosted on a DigitalOcean Droplet, with 4 GiB of RAM and 2 virtual CPUs. This specification ensures that both platforms have equal resources, making our performance comparison as fair as possible.

No Cache Systems

To assess each platform's real performance, we're conducting our tests without any cache systems in place. This approach allows us to measure the true backend performance of Medusa and Magento without the influence of caching mechanisms that could skew the results.

Product Database

Both platforms are populated with 1,000 identical products. This ensures that our tests compare like for like, with both systems handling an equal amount of data during the test scenarios.

Deployment

Medusa We used Medusa version 1.20.2 with Node 18.20.0 on our server using Docker.

Magento We used Magento Open Source 2.4.6-p4, running on PHP 8.1 and Apache

Types of Tests

To evaluate Medusa and Magento's performance, we've designed a test series that simulates different user load scenarios. Each scenario is applied to both platforms and across two distinct types of requests:

  • Single product retrieval
  • Fetching 15 Products with an Offset of 500.

This approach allows us to assess how each platform handles varying degrees of demand and specific data retrieval tasks.

Test Scenarios

  1. 50 Iterations with 1 User: This test simulates a user making 50 consecutive requests. It's designed to assess how the platforms perform under a light, consistent load, providing insight into their efficiency in handling individual user actions.
  2. 10 Users Simultaneously (for 30s): By simulating 10 users accessing the platform simultaneously, this test evaluates the platform's ability to handle multiple requests concurrently. It's a basic concurrency test, reflecting a realistic scenario for small to medium-sized eCommerce sites during low to moderate-traffic periods.
  3. 25 Users Simultaneously (for 30s): This scenario significantly increases the concurrency, testing the platforms' performance under a medium load. It simulates a more intense shopping period, such as a sale event, during which many users browse and interact with the site simultaneously.
  4. 50 Users Simultaneously (for 30s): The most demanding test, this simulates a high-traffic event, challenging the platforms' scalability and resource management under heavy load. It's indicative of peak shopping times, such as significant sales or holiday shopping rushes.

Requests Tested

  • Single Product Retrieval: Measures the platforms' response times when a user requests information for a single product.
  • Fetching 15 Products with an Offset of 500: Evaluates the platforms' performance in paginated data retrieval, a common scenario for users browsing through large collections of products.

Test: Single Product Retrieval

Magento endpoint

/rest/V1/products/[product_sku]

Medusa endpoint

/store/products/[prod_id]

50 Iterations with 1 User

a. Magento

k6-1
avg184.14 ms
min146.24 ms
med185.6 ms
max293.22 ms
p(90)202.21 ms
p(95)214.25 ms

b. Medusa

k6-2
avg29.88 ms
min17.89 ms
med26.44 ms
max80.61 ms
p(90)39.37 ms
p(95)49.52 ms

c. Magento vs Medusa

http_req_durationMagentoMedusaHow many times faster is Medusa?
avg184.14 ms29.88 ms6.16
min146.24 ms17.89 ms8.17
med185.6 ms26.44 ms7.02
max293.22 ms80.61 ms3.64
p(90)202.21 ms39.37 ms5.14
p(95)214.25 ms49.52 ms4.33
HTTP Request Duration (ms) - Single Product [Test - 50 Iterations with 1 User].svg

10 users simultaneously for 30 s

a. Magento

k6-3
avg1080 ms
min192.96 ms
med1060 ms
max4980 ms
p(90)1200 ms
p(95)1290 ms

b. Medusa

k6-4
avg154.21 ms
min56.6 ms
med149.56 ms
max410.13 ms
p(90)194.78 ms
p(95)214.06 ms

c. Magento vs Medusa

http_req_durationMagentoMedusaHow many times faster is Medusa?
avg1080 ms154.21 ms7.00
min192.96 ms56.6 ms3.41
med1060 ms149.56 ms7.09
max4980 ms410.13 ms12.14
p(90)1200 ms194.78 ms6.16
p(95)1290 ms214.06 ms6.03
HTTP Request Duration (ms) - Single Product [Test - 10 users simultaneously for 30 s] (1).svg

25 users simultaneously for 30 s

a. Magento

k6-5
avg2880 ms
min1290 ms
med2580 ms
max7060 ms
p(90)3300 ms
p(95)6150 ms

b. Medusa

k6-6
avg416.93 ms
min243.46 ms
med400.08 ms
max955.38 ms
p(90)506.95 ms
p(95)558.85 ms

c. Magento vs Medusa

http_req_durationMagentoMedusaHow many times faster is Medusa?
avg2880 ms416.93 ms6.91
min1290 ms243.46 ms5.30
med2580 ms400.08 ms6.45
max7060 ms955.38 ms7.39
p(90)3300 ms506.95 ms6.51
p(95)6150 ms558.85 ms11.01
HTTP Request Duration (ms) - Single Product [Test - 25 users simultaneously for 30 s].svg

50 users simultaneously for 30 s

a. Magento

k6-7
avg5730 ms
min2750 ms
med5430 ms
max9950 ms
p(90)8660 ms
p(95)9100 ms

b. Medusa

k6-8
avg936.39 ms
min494.78 ms
med884.96 ms
max2570 ms
p(90)1150 ms
p(95)1240 ms

c. Magento vs Medusa

http_req_durationMagentoMedusaHow many times faster is Medusa?
avg5730 ms936.39 ms6.12
min2750 ms494.78 ms5.56
med5430 ms884.96 ms6.13
max9950 ms2570 ms3.87
p(90)8660 ms1150 ms7.53
p(95)9100 ms1240 ms7.34
HTTP Request Duration (ms) - Single Product [Test - 50 users simultaneously for 30 s].svg

Test: Fetching 15 Products with an Offset of 500

Magento endpoint

/rest/V1/products?searchCriteria[pageSize]=15&searchCriteria[currentPage]=34

Medusa endpoint

/store/products?limit=15&offset=500

50 Iterations with 1 User

a. Magento

k6-9
avg441.35 ms
min357.93 ms
med442.7 ms
max545.1 ms
p(90)483.18 ms
p(95)490.76 ms

b. Medusa

k6-10
avg62.15 ms
min38.97 ms
med56.64 ms
max203.11 ms
p(90)72.19 ms
p(95)82.08 ms

c. Magento vs Medusa

http_req_durationMagentoMedusaHow many times faster is Medusa?
avg441.35 ms62.15 ms7.10
min357.93 ms38.97 ms9.18
med442.7 ms56.64 ms7.81
max545.1 ms203.11 ms2.68
p(90)483.18 ms72.19 ms6.69
p(95)490.76 ms82.08 ms5.98
HTTP Request Duration (ms) - 15 Products [Test - 50 Iterations with 1 User].svg

10 users simultaneously for 30 s

a. Magento

k6-11
avg2160 ms
min473.95 ms
med2060 ms
max6620 ms
p(90)2360 ms
p(95)2610 ms

b. Medusa

k6-12
avg460.23 ms
min166.26 ms
med439.68 ms
max1480 ms
p(90)597.98 ms
p(95)656.33 ms

c. Magento vs Medusa

http_req_durationMagentoMedusaHow many times faster is Medusa?
avg2160 ms460.23 ms4.69
min473.95 ms166.26 ms2.85
med2060 ms439.68 ms4.69
max6620 ms1480 ms4.47
p(90)2360 ms597.98 ms3.95
p(95)2610 ms656.33 ms3.98
HTTP Request Duration (ms) - 15 Products [Test - 10 users simultaneously for 30 s].svg

25 users simultaneously for 30 s

a. Magento

k6-13
avg5490 ms
min3260 ms
med5090 ms
max9660 ms
p(90)8540 ms
p(95)9070 ms

b. Medusa

k6-14
avg1160 ms
min759.49 ms
med1110 ms
max2670 ms
p(90)1350 ms
p(95)1430 ms

c. Magento vs Medusa

http_req_durationMagentoMedusaHow many times faster is Medusa?
avg5490 ms1160 ms4.73
min3260 ms759.49 ms4.29
med5090 ms1110 ms4.59
max9660 ms2670 ms3.62
p(90)8540 ms1350 ms6.33
p(95)9070 ms1430 ms6.34
HTTP Request Duration (ms) - 15 Products [Test - 25 users simultaneously for 30 s].svg

50 users simultaneously for 30 s

a. Magento

k6-15
avg14200 ms
min6810 ms
med12690 ms
max21290 ms
p(90)19940 ms
p(95)20330 ms

b. Medusa

k6-16
avg1960 ms
min489.28 ms
med1890 ms
max3700 ms
p(90)2260 ms
p(95)2900 ms

c. Magento vs Medusa

http_req_durationMagentoMedusaHow many times faster is Medusa?
avg14200 ms1960 ms7.24
min6810 ms489.28 ms13.92
med12690 ms1890 ms6.72
max21290 ms3700 ms5.76
p(90)19940 ms2260 ms8.82
p(95)20330 ms2900 ms7.01
HTTP Request Duration (ms) - 15 Products [Test - 50 users simultaneously for 30 s].svg

Summary

Our performance comparison between Magento and Medusa APIs shows Medusa's superior speed. Even though responses from Medusa's API are heavier regarding data size, Medusa's response times are undeniably much faster across the board.

In tests simulating multiple users accessing the system simultaneously, we observed a significant difference in response times and the volume of data processed. The Medusa API allowed us to execute more requests in the same timeframe, fetching more data.

95th percentile (p(95)) times from all tests reveals the significant differences in performance:

p(95)MagentoMedusaHow many times faster is Medusa?
Single Product - 50 iterations with 1 user214.25 ms49.52 ms4.33
Single Product - 10 users simultaneously for 30s1290 ms214.06 ms6.03
Single Product - 25 users simultaneously for 30s6150 ms558.85 ms11.01
Single Product - 50 users simultaneously for 30s9100 ms1240 ms7.34
15 Products - 50 iterations with 1 user490.76 ms82.08 ms5.98
15 Products - 10 users simultaneously for 30s2610 ms656.33 ms3.98
15 Products - 25 users simultaneously for 30s9070 ms1430 ms6.34
15 Products - 50 users simultaneously for 30s20330 ms2900 ms7.01
ALL TEST AVERAGE6156.88 ms891.36 ms6.50
  • For single product fetches across 50 iterations with a single user, Medusa was 4.33 times faster than Magento.
  • When the load increased to 10, 25, and 50 users simultaneously for 30 seconds, Medusa consistently outperformed Magento, being 6.03, 11.01, and 7.34 times faster, respectively.
  • The performance advantage extended to fetching 15 products simultaneously, with Medusa being 5.98 to 7.01 times faster across different user loads.

Medusa outperforms Magento, being 6.5 times faster on the same server setup. This means Medusa can efficiently handle many more requests and users with the same server resources.

In today's market, every millisecond counts towards making a sale. Choosing a fast eCommerce backend like Medusa could be the key to staying ahead.

Interested in using Medusa in your eCommerce? We're here to help you.

Let's talk about your project

Other blog posts

Medusa vs Magento: Total cost of ownership

Magento, compared to Medusa, may lead to higher long-term costs due to its licensing model and the risk associated with the gradual decline in the popularity of the PHP language...

Medusa.js Showcase: 5 Marketplaces and Stores for Inspiration

This article will introduce you to outstanding examples of eCommerce stores built with Medusa...

Tell us about your project

Got a project in mind? Let's make it happen!

placeholder

Grzegorz Tomaka

Co-CEO & Co-founder

LinkedIn icon
placeholder

Jakub Zbaski

Co-CEO & Co-founder

LinkedIn icon