All systems are operational

Past Incidents

14th February 2020

No incidents reported

13th February 2020

Buffalo Thermals, scheduled 1 month ago

We are installing additional cooling to many of our new Buffalo nodes, this will result in rolling reboots.

12th February 2020

No incidents reported

11th February 2020

IX Connectivity, scheduled 1 month ago

Maintenance to update router software on link to AMS-IX, DE-CIX and LINX, up to 15 minutes increased latency and possible disconnects expected.

Grantham UK only - 2h window.

10th February 2020

No incidents reported

9th February 2020

No incidents reported

8th February 2020

No incidents reported

7th February 2020

No incidents reported

6th February 2020

Compute Nodes lax110 node issues

We identified an issue regarding one of our nodes where vms did not have network connectivity. Upon rebooting the node to attempt to resolve this, it did not come up correctly.

We are liaising with the remote data centre and will update when we have further information.

Update: 2020-02-06 17:54 UTC

This has now been resolved.

5th February 2020

No incidents reported

4th February 2020

No incidents reported

3rd February 2020

No incidents reported

2nd February 2020

No incidents reported

1st February 2020

No incidents reported

31st January 2020

Los Angeles-1 API Endpoint AS Loop in communications between Miami and Los Angeles

We have taken our Los Angeles API endpoint down as there is currently a network routing loop causing communications on the return path from some of our locations (namely, Miami) to fail, meaning the API will wrongfully report services as offline and be unable to control them.

Traffic will be routed automatically to other API endpoints, but this may result in slower control panel performance than usual.

We've worked with our transit provider to exclude our routes from the offending upstream.

30th January 2020

No incidents reported

29th January 2020

No incidents reported

28th January 2020

Compute Nodes Buf152 outage

Buffalo node #152 was detected as frozen.

6:30 am GMT; Upon reboot it is displaying memory errors. 6:52 am GMT; Remote hands have been requested to reseat and if still failing, replace the faulty memory module(s). 7:29 am GMT; Reseating the memory has allowed the machine to boot. 8:10 am GMT; We've taken the machine down to fully test the memory to ensure this does not recur. 8:47 am GMT; Passed first test, bringing machine back online and will monitor.

27th January 2020

No incidents reported

26th January 2020

No incidents reported

25th January 2020

No incidents reported

24th January 2020

No incidents reported

23rd January 2020

No incidents reported

22nd January 2020

No incidents reported

21st January 2020

No incidents reported

20th January 2020

No incidents reported

19th January 2020

No incidents reported

18th January 2020

dal110 kernel issue

We are currently investigating a kernel panic on one of our Dallas compute nodes. We are working to fix this as soon as possible.