Incidents reported on status page for Fragment https://status.onfragment.com/ https://d1lppblt9t2x15.cloudfront.net/logos/ea04bf57aa7fca91db8e2a02f7f5f463.png https://status.onfragment.com/ en https://status.onfragment.com/ Fri, 05 Dec 2025 09:12:28 +0000 https://status.onfragment.com/#369b11572b128721d6b305d3f643dc4a8b7f31fc74ea3dd97b5549214359fd31 api.onfragment.com recovered https://status.onfragment.com/incident/780460 Fri, 05 Dec 2025 09:11:00 -0000 https://status.onfragment.com/incident/780460#7ef64f8ef3413f0a5f87f28500715f6be7752072a78c0ce61ee26f3d734ac38d Resolved. From 08:47 to 09:11 UTC, all incoming web traffic returned 500 errors. https://status.onfragment.com/ Fri, 05 Dec 2025 08:49:03 +0000 https://status.onfragment.com/#369b11572b128721d6b305d3f643dc4a8b7f31fc74ea3dd97b5549214359fd31 api.onfragment.com went down https://status.onfragment.com/incident/780460 Fri, 05 Dec 2025 08:47:00 -0000 https://status.onfragment.com/incident/780460#61909190bfcdf4579d673e6eb63f9efcc7875aaf7856e7704aabc8c1ce77aca8 Cloudflare is down. https://status.onfragment.com/incident/753686 Sun, 02 Nov 2025 04:10:27 -0000 https://status.onfragment.com/incident/753686#26979cd483b25f7da185f290fec470bb9eff8bb37901448c3200b1f6df03ce5f Maintenance completed https://status.onfragment.com/ Sun, 02 Nov 2025 04:01:54 +0000 https://status.onfragment.com/#9db4206725fc6ee22eb6152d9bc804f8cb1854ac84abb4b2159fea377e8bddc6 api.onfragment.com recovered https://status.onfragment.com/ Sun, 02 Nov 2025 04:00:55 +0000 https://status.onfragment.com/#9db4206725fc6ee22eb6152d9bc804f8cb1854ac84abb4b2159fea377e8bddc6 api.onfragment.com went down https://status.onfragment.com/incident/753686 Sun, 02 Nov 2025 04:00:27 -0000 https://status.onfragment.com/incident/753686#6f710581f4d6b51beedf271201c68294b1fc47e12a1d8b4e64d5fe242dd4ab43 Both API and web app won't be available for a few minutes starting 2nd November 4am UTC https://status.onfragment.com/incident/755283 Fri, 31 Oct 2025 12:57:00 -0000 https://status.onfragment.com/incident/755283#3ef9a8e6a9d20a7e18a5fc45c6031d8323afbb3103a59f468752205c1e27693b We have stopped the deployment and rolled-back to the previous version. We're no longer seeing dropped requests. https://status.onfragment.com/incident/755283 Fri, 31 Oct 2025 12:53:00 -0000 https://status.onfragment.com/incident/755283#3cdbe572dc1ad357087df4c7d82cd5c4ded8b14b8fba3b292f3a19963d0f5c5b We're seeing spikes of 503 requests due to a failed deployment https://status.onfragment.com/incident/736410 Fri, 03 Oct 2025 15:07:00 -0000 https://status.onfragment.com/incident/736410#7cdf261c2b65eee6f50a3a2f5016d1a1bdb07452982404db435ee286ed770ad4 A previous deployment changed some indexes to make some queries faster, but it also resulted in slower queries in other areas. We have performed a series of optimizations that should lead to faster queries overall. We've added load testing for the most common queries to catch future regressions and are monitoring timings in production. https://status.onfragment.com/incident/736410 Thu, 02 Oct 2025 14:40:00 -0000 https://status.onfragment.com/incident/736410#a1d4e9f4ccdb35d66e2b2d5d4ebb936e55d89d6ee037e05583349e5fce2c9b9a We're seing some slow queries, mostly in the observability panel. We're investigating https://status.onfragment.com/incident/725088 Tue, 16 Sep 2025 16:02:00 -0000 https://status.onfragment.com/incident/725088#d2079369e82f367af363154f31236022a28719e8d31b3b993b8ac4c85297a2ee We've identified the cause and are working on a fix. We're seeing very slow write queries at peak traffic that have bad consequences. We released a first mitigation to limit the impact of those queries and are working on understanding why those queries are slow. https://status.onfragment.com/incident/725088 Tue, 16 Sep 2025 07:56:00 -0000 https://status.onfragment.com/incident/725088#3d8b3e7db90f01d05f883e0115a75007382c79709a3e2eddaac3ff5104400172 Spikes of 503 errors during peak hours. https://status.onfragment.com/incident/723197 Fri, 12 Sep 2025 13:59:00 -0000 https://status.onfragment.com/incident/723197#66f31b960c4b0a62ac8af06cb909e7fc34f147e1284b706152aa071620c68f41 The API was not available during 2 time chunks chunk 1: from 12:34:33pm to 12:35:08pm (35 seconds) chunk 2: from 12:35:46pm to 12:36:42 (56 seconds) The first chunk was the result of an unexpectedly large analytics query from one of our customers combined with an improper thread management at the server level (blocking other queries). The second chunk was a consequence of the accumulation of requests during the first chunk that overloaded the replicas once they started processing requests again. This triggered a restart + scale up event. The whole process took 56 seconds after which situation went back to normal At 13:51pm, a fix has been deployed to prevent large queries from blocking the server. https://status.onfragment.com/ Fri, 12 Sep 2025 12:36:41 +0000 https://status.onfragment.com/#cc3f30024bc9d381c9b0b80e6caad224c63b2b153b1d795aede3003ca624b928 api.onfragment.com recovered https://status.onfragment.com/ Fri, 12 Sep 2025 12:34:52 +0000 https://status.onfragment.com/#cc3f30024bc9d381c9b0b80e6caad224c63b2b153b1d795aede3003ca624b928 api.onfragment.com went down https://status.onfragment.com/incident/614124 Sun, 06 Jul 2025 04:10:04 -0000 https://status.onfragment.com/incident/614124#62cbd95a92de1368330b34a6b9ef0d9e787354f4de80b7942346f47db99a934c Maintenance completed https://status.onfragment.com/incident/614124 Sun, 06 Jul 2025 04:00:04 -0000 https://status.onfragment.com/incident/614124#a623fd2defe4da7013c817d0693878176704a45776748c58c204695e0fba4a06 Our cloud provider will be performing some maintenance on our DB. We expect a few minutes of downtime https://status.onfragment.com/incident/604418 Sun, 22 Jun 2025 04:10:00 -0000 https://status.onfragment.com/incident/604418#1de3b00d6561c3ddfc2721181d6caacf8ac6855b63f8ab11b7e16a8a565f1b24 Maintenance completed https://status.onfragment.com/incident/604418 Sun, 22 Jun 2025 04:00:00 -0000 https://status.onfragment.com/incident/604418#83f7fecf4709c75387bfd0b5f2409e3d82c2ed0e4fd562208609a3c26b004620 Our cloud provider will be performing some maintenance on our DB. We expect a few minutes of downtime. https://status.onfragment.com/incident/598162 Fri, 06 Jun 2025 11:58:00 -0000 https://status.onfragment.com/incident/598162#f42d12e47373fae281c087a00d9e441bd6b54b200fde32ac4448661b3b30d3f5 All task inserts and updates were rejected during 11:55:57 to 11:58:02 UTC due to a incorrect migration. https://status.onfragment.com/incident/598162 Fri, 06 Jun 2025 11:56:00 -0000 https://status.onfragment.com/incident/598162#a2bb05ef46f5e9f30f46bc846b88df1eb2c83b328e7d2d8201c09c13d24e54b8 All task inserts and updates were rejected during 11:55:57 to 11:58:02 UTC due to a incorrect migration. https://status.onfragment.com/incident/596888 Thu, 05 Jun 2025 14:27:00 -0000 https://status.onfragment.com/incident/596888#79983c23e6d959579e1f0329e9e136e2bad8ad1bbf61fa680f31cac52f201b3e Migration is over -- we have seen very little impact on production. https://status.onfragment.com/incident/596888 Wed, 04 Jun 2025 11:40:00 -0000 https://status.onfragment.com/incident/596888#93b8eba035eacee8e440868a3db7aa1cfb4bd92f8ab68e344458490e2d659d23 A migration is in progress. You might see slower requests than usual. We're monitoring the migration to limit the impact. https://status.onfragment.com/incident/595581 Mon, 02 Jun 2025 13:51:00 -0000 https://status.onfragment.com/incident/595581#286b758548f15ecf5faa59f874a8438857f6ca756b47ce47b08e8a2fd6f8e5d0 We're no longer seeing dropped requests. We identified heavy load of an un-optimized dashboard query. We will migrate this query to a newer, faster query. https://status.onfragment.com/incident/595581 Mon, 02 Jun 2025 13:37:00 -0000 https://status.onfragment.com/incident/595581#5decde78b598ea41622c247caececb18c48b4007003cc283db00cb58691d7294 We're seing some dropped requests. We're investigating. https://status.onfragment.com/incident/556707 Mon, 05 May 2025 13:10:00 -0000 https://status.onfragment.com/incident/556707#cf95d46d0d73e6149da111868c920819b85b9cc2bd381c41b2871d942558b215 We reduced the load by 3x on a problematic client, hoping this would help mitigate the circumstantial issue in the future. https://status.onfragment.com/incident/556707 Mon, 05 May 2025 10:21:00 -0000 https://status.onfragment.com/incident/556707#692a49f5cafccd1db3596bdd01423d82e43e1d704e20958fb8e297fa820f363c Issue seem to come from increased usage of our reporting dashboards, we're making scaling adjustements. https://status.onfragment.com/incident/556707 Mon, 05 May 2025 08:39:00 -0000 https://status.onfragment.com/incident/556707#9715235762a5f59af2e6a203609684417058746f0184fa4717497534e6a0bad2 We're seeing some dropped requests, we're investigating. https://status.onfragment.com/incident/553806 Tue, 29 Apr 2025 14:27:00 -0000 https://status.onfragment.com/incident/553806#a14ecf10d90c32dccf943b725f132a309577158ad33835989be853771caff804 We have identified the root cause and implemented a fix. https://status.onfragment.com/incident/553806 Tue, 29 Apr 2025 13:21:00 -0000 https://status.onfragment.com/incident/553806#1414a52324db0a2f47c38014404ad5635fe60d52312875229e4bc8f9b2645110 The API has been dropping requests intermittently from 3:13pm to 3:15pm. We're investigating https://status.onfragment.com/incident/549063 Tue, 22 Apr 2025 14:49:00 -0000 https://status.onfragment.com/incident/549063#35e8b40829c9ac58c136ff578d3f984f9f690de2582a13fd0de7c08e098c4711 We haven't been able to isolate a root cause and it looks like random traffic congestion due to slightly increased load. We'll continue monitoring such spikes and make scaling adjustments if such issues happen again. https://status.onfragment.com/incident/549063 Tue, 22 Apr 2025 13:40:00 -0000 https://status.onfragment.com/incident/549063#58aaf00153b96bccf164e0ee9aa79fd7d36ab4b1a1ea7c38010071f8c204dab7 We've seen a spike of 503 errors at 2:35pm CET. We're investigating https://status.onfragment.com/incident/544870 Mon, 14 Apr 2025 08:39:00 -0000 https://status.onfragment.com/incident/544870#71eebccbd5f8384fa1edc25882591ee112918209f8919cbe07cdce390c887497 The fix has been deployed. We'll keep monitoring. https://status.onfragment.com/incident/544870 Mon, 14 Apr 2025 08:28:00 -0000 https://status.onfragment.com/incident/544870#e1c8fd31a32ac741a3c91cc100f82b2460ac99b6702ab3b096e1e22e4bdb8e62 We have confirmed the root cause and a fix is under way https://status.onfragment.com/incident/544870 Mon, 14 Apr 2025 08:17:00 -0000 https://status.onfragment.com/incident/544870#8ff1c4dc821b15ba25d7990356538564e66dce5f176962b6bd791c4fadd77f8c We're seeing random 503 errors. We've identified a potential root cause and are working on addressing the issue