2008 Terminal Server
Hi, anyone else got clients with some old 2008 R2 Terminal Servers just drop their CALs overnight and stop people logging in? We've had it affect 2 clients today.
Posted Oct 31, 2019, 9:41 AM UTC
730
We have one client still with a 2008 TS. I’ll check.
2
So strange, some dropped them. Some just wouldn't report back any CALs having been issued. Just a link saying "Generate Report"
2
Nope, our client is good.
3
Thanks for looking. Odd. These are 2 servers setups we've inherited too. Not ones we've setup, so it's strange that it's done both at the same time. One had installed updates overnight, the other hadn't too.
2
Yea that is weird. We just inherited ours as well. Good luck. If there’s anything else I can check let me know.
3
We (possibly) had the same issue: this morning 3 different clients with each their own 2008R2 TS on different locations reporting not being able to connect to the TS. AFAIK this started about 3 hours ago. I checked the servers but saw nothing amiss, checked clients, suspecting a Windows Update, but nothing there either. Even called the ISP of two clients (same ISP) to check if they made changes or something, but nothing.
And then about half an hour ago, it's all working again.
I'm clueless what happened here.
2
Is your customer's TS still working normally again?
Our customer's is still having the issue and another person just responded to the chain below saying they have the same issue.
1
No, still issues this morning.
1
Yeah same. Thanks for coming back to me.
1
I'm currently experiencing this issue with a client today too! What I eventually found was if they keep trying to login it would let them on. For some users it only took 5 tries, for others it took a bunch. They are on working for the day and the plan was to look at this further after they close. I'm sooooo glad to hear it wasn't just us and I can stop digging into a million different things it might have been. However, I really wish there was an answer to what happened here today.
My RD Licensing Manager is showing the "Generate Report" link too
2
same issue here,
Any solution?
TIA
1
My customer's issue is still present this morning.
1
Change RDP server port or use whitelisting
1
FYI:
In an attempt to figure out what is going on, I've narrowed it down to this issue being a combination of Windows 10 1903 (not fully sure, but we do have a few mac users, who are not affected by this) with Windows 2008R2 server.
Looking in the Security eventlog (example), I notice a massive amount of audit failures. Notable here is the originating workstation name frequently is "Remmina" (linux rdp client) or "Freerdp". In one series of logon failures, there was an originating IP, which (seemingly) is Ukraine based.
Account names used are "Administrator, "admin", "administrateur", "administrador" and similar common accounts.
My guess is, there's a wave of RDP hacking attempts going on, which overloads the TS, preventing genuine users to login.
Putting this to the test by IP-restricting RDP connections for some clients.
1
Colleague of mine suggested it was a clearing server issue. But never had that before.
1
Only have 1 still in production, but was working fine all day. Now the weekend, so fingers crossed what ever the issue was, it fixes itself by Monday!
1
It’s already sorted from what I can gather. Should be fine.
1
still having the issue, some users can login and some can't.
Opened a ticket with Microsoft, they had no clue what's going on, they worked on it over 3 hours, 5 AM to 8:30, seeing that about 15 users will be down all day i quickly setup a 2016 VM and directed the affected users to that VM, Microsoft took forever to troubleshoot, being that the affected users were up on the new server already i didn't let them restart the 2008 TS Server because i didn't want the user that were able to login to be down, i told them to call me back Sunday night to continue, If it takes too long i'll just drop it, since we had to upgrade the server anyway with 2008 losing support soon.
If i do have any updates i'll report back.
Thanks
2
Thanks. That would be awesome. Yeahhh, these clients have been pushed for ages to upgrade and haven’t.
1
I had considered opening a Microsoft ticket too but figured it was likely to go the way yours has so far. I'm going to start spinning up a 2016 Server for my client too. It was on our 'to do' list just not for this weekend. Oh well! Do post back here if Microsoft provides anything useful.
1
FYI:
In an attempt to figure out what is going on, I've narrowed it down to this issue being a combination of Windows 10 1903 (not fully sure, but we do have a few mac users, who are not affected by this) with Windows 2008R2 server.
Looking in the Security eventlog (example), I notice a massive amount of audit failures. Notable here is the originating workstation name frequently is "Remmina" (linux rdp client) or "Freerdp". In one series of logon failures, there was an originating IP, which (seemingly) is Ukraine based.
Account names used are "Administrator, "admin", "administrateur", "administrador" and similar common accounts.
My guess is, there's a wave of RDP hacking attempts going on, which overloads the TS, preventing genuine users to login.
Putting this to the test by IP-restricting RDP connections for some clients.
1
Did that resolve your issue?
We already have software in place to autoban IPs after 3 failed login attempts so I didn't suspect it getting hammered was the issue.
1
Oh yeah, adding to what you said about narrowing to a W10 issue... What's odd for my client is their Windows Computers (W10 and one last remaining W7) can get on fairly easily within an attempt or two. However their Thin Client users have to try several times before they get in.
1
FYI:
In an attempt to figure out what is going on, I've narrowed it down to this issue being a combination of Windows 10 1903 (not fully sure, but we do have a few mac users, who are not affected by this) with Windows 2008R2 server.
Looking in the Security eventlog (example), I notice a massive amount of audit failures. Notable here is the originating workstation name frequently is "Remmina" (linux rdp client) or "Freerdp". In one series of logon failures, there was an originating IP, which (seemingly) is Ukraine based.
Account names used are "Administrator, "admin", "administrateur", "administrador" and similar common accounts.
My guess is, there's a wave of RDP hacking attempts going on, which overloads the TS, preventing genuine users to login.
Putting this to the test by IP-restricting RDP connections for some clients.
Interesting. Thanks for that. Will check it out.
That indeed seems to have done the trick. I don't see any failed login attempts anymore.
Did you see these threads: https://old.reddit.com/r/sysadmin/comments/drfoyk/suddenly_rdp_to_several_2008r2_terminal_servers/ https://www.reddit.com/r/sysadmin/comments/dqq9pl/rdp_worm_out_in_the_wild/
I guess we got attacked by Bluekeep
Oh wow :( good fun this IT stuff. Will see what I can do for my clients.
Is that an open RDP server?
on the public Internet?
With port 3389 exposed for anyone and everyone?
In 2019?