Sean Brian Townsend is an independent computer security researcher, a member and the speaker of Ukrainian Cyber Alliance. His article discusses the results of the #FuckResponsibleDisclosure campaign. Publications in our opinion and civil society sections are not edited by InformNapalm and represent personal views of their authors.

Is it difficult to hack Ukraine? How UCA tested Ukrainian government IT systems

As a fashionable conversation subject in Ukraine, cybersecurity has now become equal to the fight against corruption. Ukrainian Cyber Alliance and independent researchers spent several weeks looking for vulnerable government systems. Is it possible to hack a website using Google? Do Russian hackers exist, or are they just a myth? How do bureaucrats respond when notified about vulnerabilities? How does the law enforcement participate in this?

I have good news and bad news for you. The good news is that the efforts of UCA and volunteers resulted in many of the existing holes, including those in critical infrastructure and military systems, being patched. The bad news is not the fact of the existence of vulnerabilities in police, military, or water supply systems itself, but the general absence of defensive capabilities in Ukrainian cyberspace, even after four years of the war and hundreds of targeted and destructive cyberattacks.

We repeatedly encounter Russian hackers and see the lack of preparedness of government institutions against their attacks. For instance, in December 2016, when reading hacked communications of pro-Russia hackers, we discovered that a mail server of the Ukrainian Interior Ministry was fully compromised. We immediately reported this to the Ministry, Cyberpolice, and SBU (Security Service of Ukraine).

Since we conduct similar activities in Russia and the occupied territories, we were not surprised. We are not “the cyber-apocalypse witnesses” and understand that with sufficient time, money, and just a bit of luck anything can be hacked. What did surprise us was the complete lack of any response. Apparently, in the Ministry, they decided that it was just a coincidence, and there would be no new attacks.

This kind of reaction was not unique. Later, we would encounter it many more times, even though it’s the worst kind of all. We are often forced to provide copies of internal documents just to demonstrate the vulnerabilities, because bureaucrats deny that their systems have been broken into. In the case of the mail server, the confirmation came in SBU’s response to the reporter of InternetUA.

The claim that none of the documents are secret is the most popular excuse. But we have no interest in secrets or even the ability to play with the valves in the water mains of Rivne and Kirovohrad. We just want to know if we can get from the outside to what is stored inside. I already told the story about how we hacked Orenburg Oblast. It started with a password to the website of a local veterinary hospital, and it ended with the full penetration of the data center of the regional government – all websites, all email, all document storage, including government communications and even an FSB (Russian security service) rack.

The classification of the documents doesn’t matter. You just need to find a tiny crack, and then you get inside and reach much more important targets.

Still, this idea was very difficult to explain to the officials of State Enterprise Energoatom and Kherson Oblast Rada who left several of their properties wide open on the Internet. There was no need to hack anything, we just had to find systems with public shares. Energoatom had four such systems. I can understand the concerns of the press office of the Ukrainian nuclear energy company, since even a rumor that nuclear power stations could be penetrated by hackers can cause panic. But this is the reality. It doesn’t matter whether it’s a contractor or a small department, secret or public documents, inside the enterprise network or on a careless employee’s flash drive – a leak is a leak.

We didn’t need to use any proper hacking tools. Only search, sometimes as simply as using Google. With access to the internal network (through a flash drive or directly – one of the systems allowed direct access into the internal network) sooner or later we would reach all parts of it. This is exactly how we were able to obtain full access to the SCADA system of one of Russian power stations. This is exactly how Russian hackers were able to briefly take down Prykarpattyaoblenergo (regional energy distribution company) and Severnaya transmission station in Kyiv. But in the case of Rivne and Kirovohrad water supply systems, hacking wasn’t even needed. Everything was publicly accessible, all customer lists, IP addresses, usernames and passwords, VPN keys, and everything else necessary to conduct a small terror attack.

You can rest easy – these specific cases immediately drew the attention of SBU. However, Kyivenergoremont (Kyiv city power system repair company) and State Service of Financial Monitoring still believe that they have no problems. And seeing which systems are accessible to the public or have already been visited by other hackers is outright depressing. Interior Ministry Academy (passwords for the website and the internal network, traces of repeated hacks, database of police officers), a server of the Kyiv Oblast National Police press service (documents, usernames and passwords, access to the internal network), Kirovohrad water supply (access to critical infrastructure), Energoatom, Kyivenergoremont, Judicial service of Ukraine, National Agency on Corruption Prevention, Interior Ministry reports (including special divisions), Kirovohrad Employment Center, Nikopol Pension Fund, etc.

Many people simply can’t comprehend that all information has its value. Lviv military recruitment office published the list of fifteen thousand people believed to be avoiding enlistment (in other words, the office just could not find them at their registered addresses). It did so on its own initiative. And not just using a public share or an FTP server, but on Facebook. Which means no one there had any idea that this is personal data protected by the law. And we also found the systems of the regional recruitment office of Zakarpattia Oblast with the database of recruits, orders, communications with the regional government and the Ministry of Defense, plans, lists, unit assignments, basically everything! This is no longer an excusable mistake, but something that should involve military prosecutors, since this is classified information, not just personal data.

In other words, Russian hackers wouldn’t even need to make an effort. Just come and take it.

The next two examples are Chernihiv and Donetsk regional administrations. In Chernihiv, they made drive shares public. After our post, they blocked access to the drives, but one of our volunteers immediately found a vulnerability on their website that allowed full access to the site and the presumably inaccessible drives. Basic rules say that public services must be completely separate from the local network. But bureaucrats can ignore those rules, right?

And the situation in Donetsk administration was even more interesting. The same volunteer found a remote control package there (WSO2 web shell) installed by other hackers. They didn’t just break into the website, but also obtained administrator rights on the server, stole all passwords of the real administrators and went deeper into the internal network. I want to point out here that this is not a regular regional administration, but a civil-military administration in the ATO zone. And the hackers visited there from Samara, which is a city in Russia, the same Russia we’re at war with. Even after our post about the hack, the site stayed like that for another week. Then either the administrators decided to pretend that nothing ever happened, or the hackers saw our post and cleaned the server, but in the end the server was reinstalled. The administration press service reported “temporary technical issues”. But when Russian hackers get inside the regional administration’s network in the ATO zone, these are not “technical issues”, but espionage activities by an enemy country.

Even if the administration’s IT reinstalled the server, this is absolutely not enough. The Russians could have gained access to other systems on the network. Locking down and reinstalling the compromised system is just the beginning. The next step is to check the whole network. In case you’ve forgotten, I’ll remind you what ANNA News and “LPR People’s Militia” went through. Their IT staff kept updating their server to the latest versions, tried to protect the hacked and now restored server, but we would come back using well-hidden entry points and hack them again. After the fourth hack, “LPR People’s Militia” took down their website and never brought it back again. All useful information was taken by us and handed over to Myrotvorets Center a year ago.

Where has CERT gone?

So, how does the government respond to this? It almost never does. Yes, the vulnerabilities get patched, sometimes SBU would conduct an investigation or a training. According to Kyiv Police, Cyberpolice helped them reinstall and protect their computers – apparently, none of their own people was capable of setting up Windows so that the systems wouldn’t end up wide open on the Internet. But in general law enforcement does what it’s supposed to do: prevent crime and catch criminals. As for the people responsible for computer security, there’s no one there. In principle, each incident has to be reported by the affected organization first, then it should be referred to the State Service of Special Communication and Information Protection, and then the Service would tell you all the stories I’m telling you right now, along with its recommendations. I don’t know what exactly the Service is busy with, but they’ve done nothing and not responded to any of the described incidents or attacks.

Moreover, CERT-UA, a division of the Service, was the primary target of our campaign. And they don’t respond to any incidents, not just the ones we reported. Everyone likes to talk about Russian hackers, how scary they are, about the coming cyber-apocalypse, and the need to increase the pace of the development of the improvement of cybersecurity. Or some other such bureaucratese. Still, after dozens of cyberattacks against Ukraine there’s not a single proper incident report. Of course, we can bring professional experts from the United States, and Cisco Talos can read some logs and publish a technical report about NotPetya, and we can translate reports authored by Microsoft and Eset, but who conducted that attack? Why? What should be done to avoid it in the future? How did it become possible? No response.

If you conduct a paper audit (as required by the law about the fundamentals of cybersecurity 2126a), you will have paper security. To create a private-government partnership, a partner is required, but it has been absent. We report a hole you can drive a truck through but only hear the same stories in response: nothing happened, it happened, but with our subsidiary, well, it happened with us, but had no negative effects, and if there were negative effects, we’ll just ignore questions or lie outright in a press release. What kind of consequences are you still waiting for, after Medoc and everything else? Cybersecurity in Ukraine is the responsibility of everyone and no one.

The Ministry of Emergency Situations publishes the plans of the layout of government communication lines. What, no one ever sets fires to manholes in our country? In Kyiv, this happens pretty regularly. Kyivstar publishes the project of a cellular network (here we should tip our hats to their security service: they responded to our Facebook post in less than two minutes – government services are very far behind, with average response time of a day or more). The Ministry of Healthcare – a vulnerability in the website. Once, CERT conducted a scan of vulnerabilities of government websites (or maybe some activists got under their skins and forced them to publish the advisory), and their list marked the website of the Expert Criminology Science and Research Center as vulnerable. It remained like that for a year, until we spoke about it in a TV interview. Then a representative of the center contacted me in a direct message and promised that they pay close attention to security. It was a very nice conversation, thank you.

In direct messages, all officials are nice and kind, and sometimes they thank us in comments to our posts. Amazingly, the deputy head of Kherson regional administration even wrote her own Facebook post. What can I tell you about that? Gratitude is of course welcome, but fixed vulnerabilities are not only welcome, but also helpful, and not just to us, but to everyone. In the end, we don’t need your gratitude, we only need you to do your jobs properly. For example, when we hacked the website of Astrakhan regional council, the council speaker screamed into the TV cameras, “we’re gonna fire everyone!” So, better start correcting your mistakes now, before you have to deal with consequences. And it can’t be done without a public discussion.

Your websites are full of “news” about your conferences, meetings, legislative initiatives, and other unreadable junk, but there’s nothing about your mistakes and what you did to correct them. Don’t pretend that you’re invincible, anyone can be hacked, but if you work on correcting your mistakes, we’ll see that you care, that you can make progress. And it also serves as a warning to everyone else. Otherwise, nothing will change.

What should be done? On punishing the innocent and rewarding the bystanders

No volunteers, hackers, experts, high wages, or strict penalties will help by themselves. We caught Zaporizhstal, and a supermarket chain, and a Kyiv utility company that published all its accounting and some kind of key in a folder called “BANK”, probably giving access to their current account. For some reason, we learn that Cyberberkut used documents the Russians stole in the Ministry of Ecology in its attack against Energoatom in a Facebook post, and that only because Energoatom wanted to avoid taking responsibility for it. Why is the Ministry silent? Or maybe you think there’s nothing interesting there? I can assure you that the reports of the inventory commission about the state of nuclear facilities made for such an interesting read that the nuclear people’s press service would have jumped out of their skins at my mention of that if that had been possible on Facebook. (By the way, nuclear safety in our country is in a pretty good shape. Some small incidents do happen, but they don’t have any consequences.)

In the Ukrainian part of the Internet, useful information can be easily dredged up. We found a notary public with all their keys that give access to registries. In August, something happened to the website of the Foreign Intelligence Service. It ran WordPress (Google remembers everything). A taxi company published logs of all its trips.

Endless boilerplate replies, weak excuses, blame games and apologies on Facebook do not provide any protection. A dozen different cybercenters (we have cybercenters everywhere, they’ll soon start popping up at local housing offices) with their empty rhetoric whose only tangible result is the disappearance of government money. Instructions and rules for underwater combat with aliens are being produced by the ton, while Russian hackers are free to run around Ukrainian networks, including civil, military, and critical infrastructure.

So, what should we do about cybersecurity? We should simplify, not complicate laws and instructions. Eliminate useless organizations. Take down useless websites never visited by anyone. It’s easier to make a reference list of district administrations than support hundreds of junk websites. Fire useless people who not only can’t do their job, but actually sabotage their employers. Start with the basics. Remove public access from SMB shares and FTP servers, disconnect public resources from internal networks, use proper passwords and two factor authentication, never click on random links. And, most importantly, if something happens, talk about it honestly, notify the Service of Special Communication, try and figure out what happened, how it happened, and who benefited from it. When you try to hide the truth, you hurt both yourselves and your country.

Formal replies and blame games won’t help defend against a large scale coordinated cyberattack. Of course, we won’t die, but we’ll have to go back to abacuses and candles. And don’t think that you’re surrounded by idiots, while you have the best IT staff, great policies and everything else. Anyone can be hacked, it’s just a matter of time, money, and motivation.