3.30am Illumination

This happened last night. At 3.30am I woke to dreaming I had a solution to our problem, well the problem that has the potential for escalating quickly at work. Here's the summary of all of that:

The Environment/Setup:



So, we have couple monitoring servers that monitors and manages the kiosks out in the field. The IPs for the monitoring servers are hard-coded within the image. Every 30 minutes the kiosk pings the gateways to check if there's any tasks waiting for it. For security. on the router end we also have the IPs setup so when the router sees incoming traffic it will allow it to pass.

The Problem:

Up until recently the monitoring servers had Class C IPs. Life was good. But then the decision was made to migrate the server OS from Windows Server 2003 to 2008 R2 AND move to Class A IPs instead. By this point the kiosk themselves are coming to end of life and we made the decision to retire the whole platform by the end of 2019. Yay!!

With the IP changes on the monitoring server, we had to change the IPs on the kiosk agent themselves. And we were able to accomplish that working hand in hand with the network folks. That particular event went down like this:

1. Week 1: network ACL adds 1 new IP to the list keeping the old ones still in
2. Week 2: migrate that one particular monitoring server that was added on the network ACL and update the agents to replace the Class C IP to Class A for that particular one.

Few months later after verifying all are functioning OK, repeat the same process for other server.

This is great for existing devices. What's not great is any devices that have to be re-imaged or new kiosk installation. Remember I said the kiosk itself is coming to an end.

Oh and the old IPs has been reclaimed. Meaning they are gone. Disappeared.

Finally the Illumination:

Since now there's no way for us to contact the kiosk in order to update the IPs on the agent, I had THE grand idea. Oh BTW, did I mention I designed the whole platform in such a way even if we considered having a tech manually update the agent there's absolutely no way to do it. You can't do "CTRL + ALT + DEL" or "CTRL + Esc"..nada. Without being able to check-in with the monitoring servers these machines are useless.

So, my theory was since we are killing the platform there's only a handful of instances where devices are completely down. How about update the agent on the devices via some type of autorun application that the tech can use? So techs will walk up to the downed device, plug in the stick drive and boom it will update the agent on them and reboot and reseal.

The Solution:

With that in mind, by 4am I am writing the different pieces of the whole solution. Ideally there has to be two instances. a) an AutoRun.inf file and b) the application that will update the agents.

Writing the code for either one are very simple. I was so excited I has having a hard time deciding whether to write the second one in VB, bash, script file among others.

By 5am, I was done with all of the coding and now time to test. I waited anxiously to drop our daughter off at the bus stop and then ran to the lab for testing.

For testing, I devised to only reboot the kiosk. So, if I plug in the USB stick and it reboots correctly then I can enable rest of the code for other changes. I could see the USB sticks gets recognized and but nothing else happens. What gives??!!

Heads down I am working away. Trying to figure out why the autorun.inf is running and nothing else is happening. After some digging I come to find out, MS has disabled the support to run any program automatically from a flash drive for security reasons for Windows 7 and upward. There's one of two ways to handle this:
1. Use some type of Autorun USB Creator
2. If there's a native application in the kiosk that the autorun.inf can call on and then the second application will pass the parameters for the actions.

Now time to take a pause and rethink. I search the company database for any USB creator that is approved and can be used. I am not going to download something from the web that is not approved within the company. When I can't find anything, I decided to hit up two of my colleagues with whom I occasionally discuss life, work and everything in-between. Ironically both are named John.

So, John A. I give him the rundown and my idea. He points out that's about the same as updating the current image and less time consuming than the extra component that I am trying to add on. Even though I am thinking few instances of devices being hard down, eventually he points out this little solution will propagate to all the field techs.

hmmmm...interesting point. Oh and we chatted about getting old and taking more care for our bodies...=)

John B conversation: I give him the run down and let him know I am already leaning on ditching this idea altogether. He reminds me the security protocol we have in place and how unlikely, impossible it would be to get any solution like that to be approved by the security team for release. What if someone decided to swap out the executable and update the .inf file. That means in order to safe guard the flash drive I will have to come up with another solution. Overall a time consuming process that likely won't be approved. AND we talked about our long weekend plans, beach and boat.

ahhh...it was a great idea putting on the engineering hat at 3.30am. By 11am, this is not a viable solution and all work around it ceased to exist.

1 comment:

Karishma said...

Hey, very nice site. I came across this on Google, and I am stoked that I did. I will definitely be coming back here more often. Wish I could add to the conversation and bring a bit more to the table, but am just taking in as much info as I can at the moment. Thanks for sharing.
Bigfat iBusinesses is a leading IT company focused on delivering real results to global businesses to help them experience certainty. Please visit us at Bigfat ibusinesses