Actyx Android app takes up all storage (20GB)

Hello everyone,

a Tablet that has been running for about two months with the Actyx app installed has taken over 20 GB of the device storage. The device storage is now full and starting Actyx returns the following error:

Code: FAILED_TO_START_NODE
Message: spawning core infrastructure

Caused by:
0: disc I/O error
1: Error code 4874: I/O error within the xShmMap method (trying to resize an existing shared-memory segment)
Please contact your IT administrator.

Did we do anything wrong? Over the two months a lot of events (little over 15k, a big share of them are discovery events) have accumulated, but the events are very small, I don’t think this is the main reason.

Hi @KuettnerAutomationLM,

unfortunately, this may be caused by a bug that was present in versions prior to v2.7.0. In certain scenarios, this issue caused the database to continuously grow while it should have compacted to clear out ephemeral data (that is cached on the node).
With v2.8.1, storage size handling is significantly improved. You can also set the size of the internal cache to control it using the node’s settings /swarm/blockCacheCount and /swarm/blockCacheSize. Setting on of these to 0 disables the cache. Be aware that this might slow data dissemination in the swarm significantly.

  • If blockCacheSize is hit, an internal garbage collection is run to get rid of the cache. It is specified in bytes.
  • blockCacheCount is the maximum number of blocks that are cached. If it is exceeded, GC is run. Nothing is cached anymore until this limit is met again due to blocks removed by GC.

You will need to experiment with your event size/frequency to find numbers that match your environment best. However, I’d recommend giving it a go with the defaults first, since 2.8.1 fixes the 'DB grows endlessly` bug.

Could you please update the devices to that version and see if it still happens? I strongly assume it won’t we’ve seen this behavior in another installation as well.

In the meantime, as a workaround:

  • Stop Actyx on the device with the DB that is growing too large.
  • Don’t stop the other devices.
  • Delete the database folder from the device (Settings -> Apps -> Actyx -> Clear Data or similar)
  • Restart Actyx on the device
  • Wait for the database repopulate with data from the swarm

This makes Actyx create a new database (which does not contain the ephemeral data that should be removed automatically). Since all data is available on all devices, you also won’t use any data that way.

After updating Actyx, the database will grow and shrink again from time to time, but sun will grow at a sane rate, i.e. corresponding in size to the events it actually contains.

Apologies for that, please let us know whether the workaround and update solve the problem!

Thank you for the fast reply. Actyx version on the tablet is 2.3.1.
Can we simply update to the newest 2.X?
We have to update all nodes at the same time, right?

We’re pretty sure updating one after another won’t lead to any problems when going from v2.3.1 to v2.8.1.

However, I’d suggest updating the tablet running out of disk space first and checking if everything still works as expected. After that, you can roll out to the remaining devices one after another.

I tried to delete the data on my local setup with server and tablet.
Both running version 2.3.1. I deleted the app data.
The problem now is I cant connect to the device again. (not authorized)
Did this happen because I deleted the data?
How do you get the tablet authorized? I’m unsure how I can get the user key from the tablet.
Both server and tablet are running in development.

When connecting with the node manager, the first user key to connect is automatically added to the authorized keys in that node. Did you create a key pair in the node manager in Node Auth?

You could just stop the other nodes, delete the DB again and connect with the node manager. If this does not work, we need to insert your key manually into the device’s database - give me a ping, I can walk you through that, then.