Skip to content

Archives: Service News

News content type for use by UW-IT Services on IT Connect

MWS training room

As of 10/1/2018, the Managed Workstation training room in Roosevelt Commons is no longer available.

How we provide the Managed Workstation service (MWS) training room has changed. See below for details.

What and When

During the first half of 2017, we refreshed the technology in the training room. There are photos of the space on the training room page. It’s a great option for any session that has 19 or fewer participants which each need a computer.

Using the MWS training room still requires a reservation—there is now a form which captures the needed details to streamline the reservation process.

Use of the room is open to any UW department and costs $60/hour. In the past, we bundled the costs associated with the training room into the MWS rate, but University practices have mandated we separate out this cost to ensure fair access and use. We do include ½ hour of our assistance per reservation; if more assistance is needed, we can provide that at the MWS consulting rate.

There are a variety of reservation practices which we have documented at https://it.uw.edu/wares/mws/training-room/#scheduling.

 

More info

You can find out more info about the training room at https://it.uw.edu/wares/mws/training-room/. There is also a link there to other technology spaces at the UW.

 

Brian Arkills

Managed Workstation service owner

 

SMBv1 disabled on NETID domain controllers

We disabled SMBv1 on all domain controllers for the NETID Windows domain.

Over the past couple months, we reached out to customers with resolvable client hostnames which were using SMBv1 to connect to the NETID domain controllers to let them know about this planned change. For somewhat obvious reasons, we didn’t want to publicize that we were still supporting a vulnerable protocol, which is why this change notification is happening after the fact.

What and When

The SMBv1 protocol was disabled on all NETID domain controllers on September 15, 2017.

As you are hopefully aware, the SMBv1 protocol has numerous security issues and vulnerabilities that have been exploited, making news headlines around the world. Microsoft and others have been recommending that SMBv1 be turned off, as it cannot be adequately patched or protected. For more info see below.

What you need to do

We were not able to contact less than a dozen customers which had unresolvable hostnames such as those handed out via DHCP. Those customers may need to update or reconfigure their computers to stop using SMBv1, and to use SMBv2 or SMBv3. How that is done will vary based on the operating system, application, etc., so you may need to contact your vendor(s) for assistance.

For Microsoft Windows clients, https://support.microsoft.com/en-gb/help/2696547/how-to-enable-and-disable-smbv1-smbv2-and-smbv3-in-windows-and-windows provides assistance on disabling and enabling SMB version support. There is a list of known non-Microsoft products which require SMBv1 here: https://blogs.technet.microsoft.com/filecab/2017/06/01/smb1-product-clearinghouse/.

More Info

Microsoft stopped supporting SMBv1 with the demise of Windows Server 2003 in 2015.

Significant security patches for SMBv1 were released in September 2016. And since September 2016, Microsoft has publicly been encouraging everyone to turn off SMBv1. As explained in that post, SMBv1 has serious design flaws that are 20 years old. These flaws most significantly allow person-in-the-middle attacks, but also permit data inspection, and significant performance degradation leading to denial of service attacks.

Ransomware that circulated earlier this year in high volume leveraged SMBv1 vulnerabilities. Those vulnerabilities were patched in March 2017 and were applied to domain controllers.

More recently, a significant denial of service attack, called SMBLoris, has been identified and affects all computer supporting the SMBv1 protocol. Microsoft has declined to patch current OSes for this attack.

 

In summary, SMBv1 is insecure and as the Microsoft PM responsible for SMB says “SMB1 isn’t good.” We could not in good conscience continue to run it on the NETID domain controllers. We encourage anyone at the UW who is still running it to prioritize retiring it.

Please note that to protect yourself from server operators who have not yet chosen to disable it, you should disable it on your clients.

Brian Arkills
Microsoft Infrastructure service manager
UW-IT

MWS security improvements

Security improvements don’t get a lot of attention because often they silently protect you. And in many cases we silently implement security improvements. But every once in a while, it is worth recognizing these security improvements, even if you don’t see them.

Here’s a recap of some of the recent security improvements MWS has made:

  • Microsoft’s Local Administrator Password Solution (LAPS) has been implemented for Managed Workstations. This solution provides and manages a complex unique password for the built-in administrator account on each managed workstation to reduce the exposure from a single managed workstation being compromised. Many years ago, the Managed Workstation service had a compromise that affected hundreds (Coreflood). This solution would have prevented that. 
  • Based on a data-driven request from the Office of Chief Information Security Officer, we recently added a firewall rule to protect managed workstations from off-campus access of Remote Desktop. Customers should first connect to the VPN before using Remote Desktop to their managed workstation. This improvement protects all UW accounts from compromise and protects your managed workstation.
  • SMBv1, an insecure protocol, was disabled across all managed workstations in April, and disabled on domain controllers last month. This helps to protect interception of your data.
  • Upgrades of the MWS file servers this year were prompted by a variety of vulnerabilities in their software. This also helps to protect interception of your data.
  • Annually, 1-2% of managed workstations have some kind of compromise, and we’ll continue to invest in security improvements to drive that down further. But when it does happen, we’ve eliminated reimaging costs to help get you back to working on a safe computer.

 

Managed Workstation FY18 rates

The preliminary FY18 rates for Managed Workstation services are now available. The annual rate change for UW-IT services was postponed this year to be effective 9/1/2017. Management Accounting & Analysis (MAA) provides final approval of rates for all cost-recovery centers at the UW, and these preliminary rates are still pending their approval.

All rates associated with the Managed Workstation service are documented at: https://it.uw.edu/service/managed-workstation-services/#Price.

Rates

The new FY18 rates are:

  • Managed Workstation rate: $30.00/workstation/month
  • Managed Workstation file storage: $.25/GB/month
  • Consulting Services: $104.43/hour
  • Managed Workstation training room: $60/hour

Changes

In conjunction with these rate changes, there are some changes to the services provided. These changes include:

  • Managed Workstation training room use is a separate rate, not included in the Managed Workstation rate. This change was mandated by University practices to ensure fair access and use.
  • There is a single file service rate, regardless of the underlying platform used.
  • Support assistance for MWS file services is now included in that rate.
  • There are a few previously defined consulting activities that we will now provide as part of the Managed Workstation rate. Notable examples include:
    • 1 hour of complimentary business needs IT consulting per year per customer account
    • 1 hour of complimentary computer hardware recommendation consulting per year per customer account
    • OS imaging for compromised managed workstation

Requests for the complimentary consulting will result in an in-person visit to discuss your needs.

More details about these changes are available at https://it.uw.edu/wares/mws/design/what-does-the-managed-workstation-rate-include/.

If you have any questions or concerns, please contact Dawn Cullerton or myself via help@uw.edu with a subject line of “FY18 MWS rate questions”.

Brian Arkills
Managed Workstation service owner
UW-IT

Migration update

99.9% of all managed workstations have been migrated to the NETID domain, and today we’ll be removing the 5 remaining workstations still in the Nebula2 domain. We greatly appreciate your partnership in both the user and computer migrations over the past couple years. Your patience while we completed this large undertaking has been amazing.

I’m also happy to report that new computers can be enabled for Managed Workstation services without the Nebula2 domain. https://it.uw.edu/wares/mws/hardware/adding-computer/ documents the steps needed to do that—it’s available today. In a nutshell, you now can “claim” your computer before joining it to the domain.

If you join a computer to the NETID domain without first completing that step, you’ll end up with a workstation that isn’t usable until someone on the NETID domain service team has intervened (send an email to help@uw.edu if you end up in this situation). This behavior is part of the design of the NETID domain service, and is a consequence of sharing the NETID domain with hundreds of other UW organizations. We are working on a more streamlined way to enable the workflow, leveraging the Managed Workstation imaging process. For resource constraint reasons, that work has had to wait while we focused on workstation migrations, but I’m hopeful we’ll have an even better option in the near future.

Now that the users and computers are in the NETID domain, there will be some minor planned outages over the next month to complete migration of remaining infrastructure. We’ll also start adopting the optional capabilities the NETID domain service provides, which was part of the value proposition behind undertaking this migration. An early candidate is the Local Admin Password Solution (LAPS) feature (https://it.uw.edu/wares/msinf/ous/laps/), which will reduce the risk to all Managed Workstation customers from a single computer being compromised. We’ll share more about that when we’re ready to release it.

Brian Arkills
Managed Workstation service owner

NETID domain controller upgrades: 8/2 – 8/28/2017

Several changes are planned for the NETID domain service.

What:
All NETID domain controllers (DCs) will be replaced with new servers running Windows Server 2016. An additional design change will happen during this process of relying on InCommon CA issued certificates for LDAPS access, replacing the existing design that leverages UW CA issued certificates.

When:
8/2/2017: First new WS2016 DC promoted
8/7 – 8/9/2017: 4 new WS2016 DCs promoted and 4 existing WS2012R2 DCs demoted, handled in a +1 new DC, -1 old DC fashion
8/28/2017: Last WS2012R2 DC demoted

What you need to do:
If you have an application or code which relies on the NETID domain service, you may need to adjust its configuration.

Known problems which your application may have include:
-if it does not automatically use the Microsoft DC locator process, but instead hard-codes domain controller names or caches domain controller names for an inordinate period of time
-your system does not trust the InCommon CA

It’s also worth noting that if you have system firewalls that do not follow the published NETID domain service firewall guidance, https://it.uw.edu/wares/msinf/authn/firewalls-with-netid-domain/, you may need to adjust your firewalls.

We have purposely delayed the last WS2012R2 DC demotion for several weeks to allow customers to discover and address unknown problems with their applications.

More info:
All Windows computers joined to the NETID domain are configured via domain group policy to trust the InCommon CA, which accounts for ~99% of all systems which perform LDAPS operations with the NETID domain service. However, non-Windows systems and Windows systems in other domains which trust the NETID domain may not be configured to trust the InCommon CA. Whether they are or not is subject to the platform, vendor defaults, and system operator configuration. If your system does not trust the InCommon CA, you’ll need to configure it to do so. More information about the InCommon CA and UW’s use of it is at: https://it.uw.edu/service/certificate-services/.

All Windows computers use the Microsoft DC locator process. Non-Windows computers generally do not, although there are exceptions. If your system does not automatically locate domain controllers, you may need to manually configure it and/or take actions that clear any cached information.

After the last WS2012R2 DC is demoted, we also plan to raise the domain and forest functional level to WS2016.

If you have questions, concerns, or encounter problems during these changes, please contact us by sending email to help@uw.edu with “MI DC changes” somewhere in the subject line.

Brian Arkills
Microsoft Infrastructure service manager
UW-IT

Workstation migration to NetID domain update

Approximately 2,428 managed workstations, or about 78%, have been migrated from the nebula2 domain to the NetID domain. We will continue to attempt to migrate every managed workstation at 5:00 pm daily.

We are still unable to offer a self-service option for workstation migrations at this time. The technical issues that may be encountered are unlikely to lead to a positive experience.

What’s Next:

Migration are still taking place at 12:00 pm (noon) and 5:00 pm Monday – Friday for any non-migrated workstation.

We encourage customers to send us a migration request via help@uw.edu with “MWS Migration” in the subject line so we can migrate your managed workstation during this period.

Monday 7/10/2017 we will begin reaching out to Technical Contacts with non-migrated workstations to facilitate the migrations.

What do you need to know?

Computers that have not scheduled a migration date by 7/31 will be removed from the nebula2 domain.

If you have any questions, please send an email to help@uw.edu with “MWS Migration” in the subject line.

Dawn Cullerton
Service Manager
Managed Workstation
UW-IT

Workstation migration update

Approximately 2200 managed workstations, or about 66%, have been migrated. Every non-migrated managed workstation has been a candidate for nightly migration and has been attempted at least 3 times. At any given point only 75% of managed workstations are manageable over the network, so customers will need to work with us for further progress to occur.

If you have questions about the workstation migration, a FAQ is available and updated regularly.

There were a number of problems over the last 3 weeks, some with our workstation migration approach, others related to unanticipated file service issues. We’re sorry for any inconveniences this may have caused. We’ve worked hard to address these issues and adjust our approach.

We’d love to provide a self-service option for workstation migration, but the technical details make it extremely unlikely to lead to a positive experience.

What’s Next:

During the Workday go-live period, June 16th – 30th, we will not initiate any mass migrations of non-migrated workstations. However, our offer to schedule noon migrations on a given date for any non-migrated workstation still stands during that period.

We encourage customers to send us a migration request via help@uw.edu with “MWS Migration” in the subject line so we can migrate your managed workstation during that period. Department contacts will get a list of non-migrated workstations in their department via an email later today.

After June 30, we’ll go back to attempting to migrate all remaining non-migrated workstations each night. Beginning the week of July 3, we will begin to contact customers with an non-migrated workstation to facilitate migration.

Unmigrated computers left in late July will largely be those which are not regularly on the UW network. Computers in this state are not a good candidate for the Managed Workstation service, as they receive very little of the regular value we provide. We’d like to better understand that choice and whether there is a better option needed, so we invite customers who find themselves in that situation to have a dialog with us. Feel free to send an email to help@uw.edu, or start a dialog on mws-discuss@uw.edu.

Workstation migration FAQ

This page has frequently asked questions about the workstation migration from Nebula2 to the NETID domain.

Index of questions:

Does my managed workstation need to be migrated?

Yes. Every managed workstation must be migrated.

My computer was scheduled to be migrated, but wasn’t. When will it be migrated to the NetID domain?

We are aware that some computers were not migrated on their scheduled date. The computer was either not on the network or had some other issue which prevented migration. We will re-attempt to migrate those computers. If the computer is not on the UW network, then you should get it on the UW network. If you have power settings which put your computer to sleep, you should disable those settings until your computer is migrated. You can request to have your computer migrated at noon on any day–send an email to help@uw.edu with “MWS Migration” in the subject.

How do I find my computer name?

There are a few ways you can do this:

  • Go to Start > Control Panel > System and Security > System. Your computer name will be listed about halfway down the window
  • On a Windows 10 computer, you can type in the word “About” (No quotes) in the Cortana Search field and new window will open. At the top of the window, your computer name will appear next to “PC name”.
  • Open a command line (Type cmd in the search box, and hit enter). A command line screen will open. Type the word “hostname”  (No quotes) and hit the enter. Your computer name will appear underneath the command line.
  • Go to the login banner on your computer. From there, select “Other User”. In the user name field, type ” .\ ” (No quotes). At the bottom of the login fields, it should show “Sign in to:” followed by the computer name.

How do I know if my computer was migrated to the NetID domain?

Go to Start > Control Panel > System and Security > System. Your computer’s domain name will be listed near the bottom of the window. The new domain name when your computer has been migrated will be: netid.washington.edu (The old domain was nebula2.washington.edu)

How will I Remote Desktop Connect to my computer after it has been migrated?

  • You must use a VPN when you are off the UW Network
  • Then use Remote Desktop Connect with the following computer naming convention: computername.clients.uw.edu . In the screenshot example above, the full computer name is: aerosedanwin10.clients.uw.edu

What about managed workstations that are NOT physically on campus?

We expect there will be some number of computers which we can’t migrate automatically, but we encourage you to try to get the managed workstation onto the UW network before we retire the Nebula2 domain.

What do you mean by ‘on the UW network’? Is it better for a computer to be on campus and directly plugged into the network, or is the VPN an option for the migration?

The computer needs to be remotely reachable. Unfortunately, being connected via the VPN doesn’t meet that criteria. So yes, being physically at a UW campus and either plugged in or on UW wireless is needed.

For computers which can not get on the UW network during this period, will there be a manual option to migrate to the new domain?

There will not be a self-service option to migrate a computer that is not on the UW network. There are too many potential problems for that to be a positive experience.

We do have a plan for migrating computers which are not migrated before we retire the Nebula2 domain; see the question regarding more about the Nebula2 retirement.

Many of our people will be out of the country at this time.  Do you want them to notify you?

No, customers don’t need to contact us if they know their computer won’t be available when scheduled—in fact, we advise against it. We will contact customers when we find a computer isn’t available and try to find a solution.

What if my managed workstation is not migrated?

Any managed workstation that is not migrated to the NETID domain by the time we need to shut down the Nebula2 domain may need intervention to be usable. We will make every effort to contact customers to work out a solution before that time.

We do have a plan for migrating computers which are not migrated before we retire the Nebula2 domain; see the question regarding more about the Nebula2 retirement.

When will the Nebula2 domain go away?

We are currently anticipating that will happen in late July to August.

Can you say more about the Nebula2 domain retirement?

We plan to retire the Nebula2 domain in late July or early August. After we have retired the Nebula2 domain, customers that have a non-migrated workstation can anticipate the following:

  • While the computer is not on the UW network, it will continue to work unaffected. While off the UW network, users of these workstations use “cached credentials” to log on. If they connect to the VPN to get to file services or updates, there are some minor impacts:
    • updates will fail because they are directed to use a server in the Nebula2 domain which will no longer be available
    • if they go to a location on I: to which they have never gone previously, it is likely to fail
  • When these computers return to the UW network, they will have a long boot/startup time and will eventually get a “trust failed” error message. These computers will need to be manually moved to the NetID domain by Managed Workstation staff. As a temporary workaround to return that workstation to temporary use, the network can be pulled at computer boot and reattached after boot.

For customers still needing to have their workstation migrated to the NetID domain send an email to help@uw.edu, and in the subject line it should read “MWS Migration request”

What is the impact to me when my workstation is migrated?

There are two user impacts of this work:

  • computers that are being migrated will be rebooted. A second reboot will follow the first, approximately 15-45 minutes later.
  • the computer’s name will change from existingname.clients.nebula2.washington.edu to existingname.clients.uw.edu

After a workstation is migrated, users will continue to log into their managed workstation with their NETID user, and there is no other impact.

Will there be any sort of DNS aliasing to map host names from clients.nebula2.washington.edu to clients.uw.edu?

No, we don’t really have an option to do this for reasons tied to the technical details.

Why are we migrating to the NETID domain?

  • To reduce costs by consolidating the infrastructure needed to provide you with a managed workstation
  • To leverage improved capabilities provided by the NETID domain

Is there a change to how we build new computers?

For now, new computer builds will continue as is.

In other words, you’ll join them to the Nebula2 domain, with the added step that they will need to be migrated to the NETID domain after that. When you make a claim request, we should be able to migrate at that time, but if not, after the announced 2 week period, we’ll also regularly batch up Nebula2 computers on a nightly (or even more regular) basis and migrate them until it is time to pull down the Nebula2 domain. Of course, one of the things that needs to happen before we can remove the Nebula2 domain is delivering this new approach to computer builds & claiming.

We recognize this isn’t ideal, but we didn’t want to delay other progress to complete that detail, nor did we want to put additional new burdens on customers.

And to share a little more info to give you an idea of why we think it is worth waiting a little longer, our design for the new approach should be significantly improved . If you use lite-touch for builds, it will collect the needed “claiming” info up-front and join it to the domain. That will eliminate the need to talk to us to on-board a new computer, which we think is a win-win outcome.

We will announce this future planned change.

How do I use the NetID VPN after my workstation has been migrated?

Use the instructions on the MWS VPN Service page for configuring and using the NetID VPN

I got a notice about the Managed Workstation migration, but I don’t have a managed workstation. Can I stop getting notices?

You may not have a managed workstation, and you’ll need to talk to your sponsoring organization to adjust its eligibility group to better represent who actually uses the services we provide. As a service, we can’t arbitrate or override decisions made external to our service.

If you need help identifying which organization has made you eligible, we can help with that and give you the organization contacts. Send an email to help@uw.edu with ‘my MWS eligibility org’ in the subject.

You’ll then need to speak to those contacts about adjusting the Managed Workstation eligibility group they provided to us to exclude you.

You should be forewarned that depending on how eligibility is removed it may result in loss of access to services provided by Managed Workstation. We’d be happy to discuss eligibility further with the organization contacts to meet your organization’s needs.

See https://it.uw.edu/wares/nebula/adding-users/customer-accounts/#eligibilityGroups for more on the topic of Managed Workstation eligibility.

My workstation has an error and I can’t login: “The security database on the server does not have a computer account for this workstation trust relationship.” What should I do?

Rebooting your computer up to two times is the best first action to take.

If the error persists, contact UW-IT and let them know your managed workstation is experiencing an incident due to “invalid SPNs” from the migration. We’ll prioritize fixing your workstation.

This known error happens occasionally as part of the migration, and we’re working hard on finding ways to both prevent it and proactively detect and fix it. If you run into this error, we know the fix and can get you back up and running quickly.

Workstation migration – user notice

Managed Workstation is migrating workstations from the Nebula2 domain to the NETID domain. The impact to you is very low. Please ensure your managed workstation is on the UW network during the noted period.

What and When:

Beginning Monday, May 22nd at 5pm, and every night through Friday, June 2nd, we will attempt to automatically migrate managed workstations to the NETID domain. Any given managed workstation will be scheduled for one night during this period, as documented at https://it.uw.edu/wares/nebula/news/migration/schedule/.

Computers which are unavailable will be retried on every successive night during this period until successfully migrated.

There are two noticeable impacts of this work:

  • computers that are being migrated will be rebooted, with a visible 5 minute warning message when the migration agent is ready to initiate the reboot. A second reboot will follow the first, approximately 15-45 minutes later.
  • the computer’s name will change from existingname.clients.nebula2.washington.edu to existingname.clients.uw.edu

After a workstation is migrated, you will continue to log into your managed workstation with your NETID user account, and there is no other impact.

What you need to do:

Please plan to ensure your managed workstation is turned on and on the UW network on the night it is scheduled for migration. If your managed workstation is not located on the UW network, please bring it in and leave it on the network for its scheduled night.

If your managed workstation can not get on the UW network during this period, we will contact you to work out a solution, but it will require a little more effort.

If you remote desktop to your managed workstation, please adjust your practices after your computer is migrated:

  1. First, connect to the VPN,
  2. Then, remote desktop to your computer’s new fully qualified name (existingname.clients.uw.edu).

More info:

If you have a question about this change, please first see if your question is addressed in the frequently asked questions: https://it.uw.edu/wares/nebula/news/migration/faq/.

UW-IT has a lot of experience with this type of change, and the number of issues caused by this type of change are extremely low. However, if there are problems which result in an unusable workstation, we will treat it as an urgent issue and prioritize returning the workstation to service. You can call 221-5000 or send an email to help@uw.edu and let the UW-IT Service Center know that you are experiencing an incident with your managed workstation.

If you have a significant scheduling issue during this 2-week period, and need your managed workstation to not be migrated, please let us know and we will try to accommodate you.