Incident response: collect the evidence before cleaning up
The natural reflex in front of a compromised machine is to restore it as fast as possible. It is also the surest way to destroy what would have shown how the attacker got in.
RGAIX team · · 8 min read
What immediate cleaning destroys
Rebooting, deleting suspicious files and reinstalling wipes, within minutes, the processes in memory, the open connections, the access timestamps and sometimes the logs themselves. Without those, the entry vector stays unknown.
An unknown vector means a likely re-exploitation. A showcase site restored without investigation can be compromised again in under two days, through the same door, left open.
The steps in the right order
- Isolate the machine from the network, without shutting it down: cut the access, not the memory.
- Take a copy of the state: processes, connections, scheduled jobs, recent files.
- Save the logs off the machine, before any modification.
- Only then: revoke access, fix the flaw, restore.
Look for the vector, not only the symptom
Finding a mining process and deleting it does not answer the useful question: how did it get there? The vector can be read in the access logs, in the hour before the file appeared.
- An unusual request just before the first suspicious file was written.
- A successful authentication from an address never seen before.
- A vulnerable dependency whose version matches a recent public advisory.
- An exposed service that should never have been publicly reachable.
Treat a repeat as information
If the incident happens again after remediation, the fix did not hold. Two causes dominate: the flaw was not the one you thought, or the fix was overwritten by a later deployment because it had been applied by hand on the server.
Hence a simple rule: every remediation is versioned. An uncommitted fix is considered non-existent, because it will disappear at the next release without anyone noticing.
Write the playbook during, not after
The commands used, the traps met and the order of decisions are written down as the work happens. Reconstructed three weeks later, they lose exactly what made them useful: the access details and the dead ends.
That document also serves internal communication and, where the context requires it, notification obligations. It is easier to sort recorded facts than to find them afterwards.
Restoring service is an objective; understanding the entry point is a condition. Doing the second after the first means doing both, twice.
Related pages
IT troubleshooting & emergencies
Your production is down.
Discover the service ↗READ NEXTAPIs: evolving without breaking your integrations
A published API is a contract. Evolving it takes fewer versions than discipline about what is guaranteed.
Read the article ↗READ NEXTHardening an exposed server: the minimal list that changes everything
Reducing the exposed surface costs little and removes most attempts. Six measures, in order, and one frequent trap.
Read the article ↗Let’s talk about what is blocking you.
An emergency, a profile to find or a project to structure: let’s start with your need.
