When cyberattacks happen, most of us only hear reports from the media about what the FBI might be doing or how the company that was attacked is coping with it. We don’t often get a chance to hear from those on the front lines—from the businesses who were affected or from those who helped those businesses get back up and running.
Luckily, we had just such an opportunity recently, when Jay Tipton, CEO and Owner of Technology Specialists, appeared as a guest on the Cyber Crime Lab Podcast. Jay was one of the 50 managed service providers (MSPs) who were affected and he and his team worked day and night to clean workstations and servers and get his clients back in business.
To better understand what Jay shared, we need to know the facts of the case first.
The Kaseya VSA Ransomware Attack
Even those familiar with the basics of technology might not know what Kaseya or VSA mean.
Kaseya is a software company headquartered in Dublin that offers a framework for maintaining and managing IT infrastructure. The products it offers, including one called VSA, are used by MSPs around the world.
Kaseya VSA is a remote monitoring and management (RMM), endpoint management, and network monitoring solution.
Ransomware is a type of malicious software designed to block access to a computer system until a sum of money is paid. In the case of this attack, $70M in BTC was demanded by the attackers.
This particular ransomware attack was probably initiated by a gang known as REvil, which injected code into VSA.
What it Was Like on the Ground
Jay was at a client’s, working on a laptop, and saw a couple of Microsoft products close themselves before he signed off. He thought it might be a standard program bug. But as he headed back to the office, he spoke to one of his team who told him that multiple client calls coming in to say that their computers were down.
When he got back to the office, Jay saw ransomware on one of the computers and went straight into Technology Specialists’ network operation center (NOC) and literally started pulling plugs and turning things off until he could figure out what was going on.
Over the next few hours, it became clear that all his clients had their data encrypted as part of the attack and he had to fend off angry customers who wanted to hold him accountable.
“You go from blaming yourself, to thinking of blaming others, to taking full responsibility, being totally numb, and not being able to do anything,” Jay said. He worked for almost two days straight before collapsing onto one of the company couches. He and his top engineer logged almost 500 hours each in the four weeks that followed.
During this time, two things happened that Jay and his customers had no say in:
- Kaseya refused to pay the ransom
- The FBI acquired a decryption key that it refused to share with Kaseya
The purpose of this article isn’t to critique either Kaseya or the FBI for their actions (that’s already been done) but to give context to what Jay and his team had to do. In the absence of the decrypt key, Jay offered a simple clean and restore of all the workstations and servers affected. This was an active move, as it meant not waiting for a decrypt key—which would take everything “back to normal”—but instead got companies off on the best foot they could manage with whatever backups they had in place.
Since Technology Specialists was itself affected by the attack, they had to find an old server that had contact information for clients to start making appointments to get the hardware fixed.
When they got started, Jay was overwhelmed by the support from clients and team members who pitched in with help—be it time or food. He even had ex-business partners and employees come in to help.
It took six weeks after the first day of the initial attack for all of Jay’s clients to be fully functional.
Prevention?
Jay notes that despite using industry best practices for his clients including two-factor authentication (2FA) on numerous applications, a vulnerability in software was still exploited. That’s something we have to become increasingly aware of: that despite our best efforts and security measures, it’s likely to be a question not of if but when we deal with a cyberattack that affects us or our businesses.
With that inevitability in mind, Jay and his team have been putting together services that can respond more robustly to future attacks. Jay found that at some points during the attack there was so much information coming in and so little frame of reference to make the right decision, that he simply froze. Unable to make decisions, he wasn’t able to help anyone.
This situation will be remedied in the future. Veterans of the Kaseya attack will fly out by helicopter, if necessary, to more remote clients to help them with the decision-making process that Jay had to struggle with in July and August 2021. As Jay learned, “winning” in this scenario wasn’t about waiting for the authorities to “do something” but about finding a way to communicate with his clients and get a plan of action in place. It was that “can-do” attitude that ensured that Jay kept all but one of his 50 clients, some of whom had been there from the very beginning, in 1998, when Jay started the company. His actions under pressure are a helpful guide for anyone navigating a business crisis, particularly one as traumatic as a cyberattack.
If the worst happens, you’re going to want the best financial, legal, and technical support to get you back up and running again. With cyber insurance from DataStream, we find the most comprehensive insurance coverage on the market alongside critical post-incident customer support.
Click here to learn more about how we can help secure your business data!