To PowerShell or Not

Friday, December 17, 2010 / Posted by Luke Puplett /

To pull down, sort out and import the guide data for vuPlan.tv I thought it would be a good excuse to use PowerShell. I’ve been wanting to use this for ages, to get up-to-speed with what the future of scripting on MS Windows looks like.

I hated every moment of it.

This simple script took more than a whole day to write.

#
#
Looks for the latest downloaded .zip file and extracts its contents.
#

$random = new-object -typename System.Random

$dataRoot = "C:\DATA"
$sourceFolder = $dataRoot + "\EPG"
$extractRoot = $dataRoot + "\EPG\TEMP-" + $random.Next(100)
$dataRepository = $dataRoot + "\EPG\HEAP"
$unzipToolPath = $dataRoot + "\unzip.exe"
$importerToolPath = "C:\Users\Public\Documents\~Main\Vuplan\Evoq.Vuplan.Data.Epg.Importer\Evoq.Vuplan.Data.Epg.Importer\bin\Release\import.exe"

clear

set
-location $sourceFolder
#$latestFiles = get-childitem | where { $_.Name -like "*TV*" -and $_.LastWriteTime.Date -eq [DateTime]::Now.Date }
$files = get-childitem | where { $_.PSIsContainer -eq $false }
$latestFiles = $files | sort -property @{Expression={ $_.LastWriteTime }; Ascending=$false}
$latestZip = $latestFiles[0]
echo
"Extracting " + $latestZip.FullName + " to " + $extractRoot
md
$extractRoot

# This little chunk of code unzips using Windows shell but problems with the feckin .zip file mean
#
that some files extract twice and invoke a UI prompt for overwrite-replace.
#

#
$shell = new-object -com shell.application
#
$zipFile = $shell.namespace($latestZip.FullName)
#
$extractTo = $shell.namespace($extractRoot)
#
$extractTo.Copyhere($zipFile.Items())

#foreach($item in $extractTo.Items) {
#

#
$item
#
}

$processArgs = "-o " + $latestZip.FullName + " -d " + $extractRoot
$startInfo = new-object System.Diagnostics.ProcessStartInfo($unzipToolPath, $processArgs)
$startInfo.UseShellExecute = $false

$unzipProcess = new-object System.Diagnostics.Process;
$unzipProcess.StartInfo = $startInfo
$startInfo
echo
"Starting process."
$unzipProcess.Start()

while(!$unzipProcess.HasExited)
{
echo
"Waiting for process to complete..."
[System.Threading.Thread]
::Sleep(1000)
}

$sourceDirectoryInfo = new-object System.IO.DirectoryInfo($extractRoot)
foreach ($item in $sourceDirectoryInfo.GetFiles())
{
$destinationPath = [String]::Concat($dataRepository, "\", $item.Name)

if ([System.IO.File]::Exists($destinationPath) -ne $true)
{
"Copying " + $item.Name + " to the heap..."

[System.IO.File]
::Copy($item.FullName, $destinationPath)
}

if ($item.Extension -eq ".xml")
{
$processArgs = $item.FullName
$startInfo = new-object System.Diagnostics.ProcessStartInfo($importerToolPath, $processArgs)
$startInfo.UseShellExecute = $false

$importProcess = new-object System.Diagnostics.Process;
$importProcess.StartInfo = $startInfo
$startInfo
echo
"Starting import."
$importProcess.Start()

while(!$importProcess.HasExited)
{
"[" + $item.Name + "]" + " Waiting for import to complete... " + ([DateTime]::Now - $importProcess.StartTime)
[System.Threading.Thread]
::Sleep(1000)
}
}
}

$sourceDirectoryInfo.Delete($true)



It was such a disappointment. I’m still disappointed as I type this. Being a scripting environment for the .NET Framework, I thought it’d be so cool. The power of the .NET Framework and the freedom of scripting.



But the Integrated Scripting Environment (ISE) is just some crappy cobbled-together tool written in not much more time than it took me to write my script here. It even uses the atrocious code editor from the Expression suite and not the awesome editor that Noah Richards and his team made for Visual Studio 2010.



But worst of all, it has no IntelliSense. Can you imagine?



Apparently there’s PowerGUI that is a non-Microsoft ISE that does have autocomplete (which is an absolute must when you’ve all of the .NET Framework at your disposal and there’s no compiler to warn as you go), but I read something about a guy having a problem with a script which was down to the PowerGUI editor getting confused over some combination of slashes and dots, which put me right off.



So I punched my code into the MS ISE, almost single fingering the keys to ensure it was all correct, like I was back in the 1980s with my Sharp MZ-700.



And slowly my will to live drained from me.



I even read some dude praising it because all he needed was Notepad and Google to be a master of his universe. Not out of my pocket does an engineer work with such inefficiency. Would you take the cab if you saw that the driver was peddling the car through a hole in the floor?



Next time, I will do what I usually do and write a full blown console application in a tenth of the time.



If you’re a .NET developer, Hell, if you’re a sys admin (as I was for years), I’d just use Visual Studio all the way. Skip those cmdlets.



Get properly organised by writing and adding to an assembly of reusable components and classes to help you do your admin tasks. Get the whole department adding to a super cool library of administrative joy.



I came into programming from a sys admin background. I worked for a very large organisation with tens of thousands of computers. I saw the same code across so many scripts, it was a terrible duplication of effort.



So I cultivated a library of fine-grained admin tools in a COM component that was deployed to the PC estate and called from the scripts, a core set of reusable functionality. Then later I integrated it with a Microsoft Excel add-in. That’s right, sys admins could build out their scripts from a report in Excel, drag and drop, cut and paste. Done.



Anyway, I’m going off on a tangent. The benefits of PowerShell from my noob viewpoint are, simple visual remotability and a focus for vendors to create simple APIs (by way of custom cmdlets).



These things are important for the daily activities of a busy sys admin performing live commands, sure, but if its a automated or long scripted process you need to write, its not ideal.



If you’re already a programmer, then just write a proper applet in the superior tools you know and love and share it with your team. And if you’re not a programmer, skip scripting and learn C#.



Coding in Visual Studio will blow your mind and make you a hero.



Luke

Labels: ,

9 comments:

Comment by jsnover on Friday, December 17, 2010

Howdy Luke!

First let me start out by thanking you for the critism. I find it VERY helpful when people take the time to articulate what they don't like and why they don't like it or where their PowerShell experience sucks. I agree with a number of the points you made and looking at the some of the code you had to write made me blanch. Your specificity helps me make sure that we will fix these things going forward.

Thanks.
Full stop.

Next.

It doesn't have to be as bad an experience as you had.

My experience is that we always start out with a language by writing our old programs with a new syntax. It take a while to begin to think in the new idioms of the language and use the new mechanisms. Who wants to take the time to learn them when your hair is on fire and you are trying to get something done?

From your example, it is clear that you are a .NET programmer. You were probably lulled into a wrong set of expectations by the fact that various aspects of PowerShell's syntax are very similar to C#. I can see where those expectations would produce a frustrating experience.

I think that if you wanted to spend some time learning some of the PowerShell cmdlets and idioms, that you'd find that it is amazingly productive for a class of problems.

All that said, at the end of the day, you should use the tools that you can succeed with. We believe in customers getting stuff done, not pushing a technological agenda.

Thanks again for the details - I appreciate them.

Have a happy holiday!

Jeffrey Snover [MSFT]
Distinguished Engineer
Visit the Windows PowerShell Team blog at: http://blogs.msdn.com/PowerShell
Visit the Windows PowerShell ScriptCenter at: http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx

Anonymous on Saturday, December 18, 2010

Very first line is a good prove you have not PowerShell mindset. $random = blah means you write C# in PowerShell, is as wrong as writing VBScript there. You hardly use any cmdlet (Get-Random, Start-Sleep, Start-Process, Get-Date, Test-Path) and expect PowerShell to help you with that? ;)

Tab completion is all you need (usually). And ISE was never made to replace VS, it was made to replace notepad.exe. So basically: your experience with PowerShell is still close to 0. You hardly used it. You just re-written C# and changed the syntax a bit to match PowerShell scripting language. So programmers.. sure, probably don't need PowerShell, compiled code is usually quicker than any script. Admins - well, hell-no. But first of all they need to use cmdlets whenever possible, and fall into .NET only when necessary. Quite opposite to what you did (in my opinion).

Comment by Luke Puplett on Sunday, December 19, 2010

@Jeffrey –

Dear Jeffrey, many thanks for taking the time to make such considered comments. You said that learning in the context of trying to get somewhere in a hurry will evoke frustration, and I wholly agree; that’s a very important point.

Before posting, I did order a PowerShell book. Being a noob in anything is pretty tough, especially when you’re not used to feeling helpless and inadequate, and if I didn’t try to learn and read the manual, then I’d only have myself to blame.

I think my high expectations were not of the language, but more of the tooling experience and the achievable productivity levels. The language and its new paradigms are laudable, and I could well imagine growing to love it, but adoption and friction being immiscible, this won’t happen with the current ISE.

As a programmer I literally live in Visual Studio. The VS IDE is a wonderful work of software engineering and a testament to its makers. When cutting code, I’m very efficient, made more so by some 3rd party tools.

As a sys admin, I lived in DOS and Notepad, and often scripted VBScript out of Visual Basic or the Office VBA editor. So in 2011, I expect Microsoft to be equipping sys admins with tools of the quality of Visual Studio, focused on heavy textual input and replete with an add-in marketplace and extensibility story.

If you’ll allow me to be cynical, I think this is because Microsoft understands the critical role of developers in their ecosystem. Developers add value, while sys admins do not. However, Microsoft could turn sys admins into developers if they brought them under the wing of Visual Studio.

I think your mission, should you choose to accept it, is to put Visual Studio on the desktops of all technicians, and .NET Framework wherever there is a microchip.

My other message was for sys admins to think smarter, more collaboratively, to extend their skills and get some social kudos from their team-mates while they’re at it, and be better positioned to negotiate a pay rise.

Merry Christmas!!

Luke

Comment by Luke Puplett on Sunday, December 19, 2010

@becomelotr --

"You just re-written C# and changed the syntax a bit to match PowerShell scripting language." That I did!

Given more time, I'm sure I'd pickup the short-cuts and even more so when your seeing lots of other people's scripts each day, and you see things you didn't know were possible.

Thanks for commenting - I should write more controversial material in future ;)

Comment by jsnover on Monday, December 20, 2010

"adoption and friction being immiscible"
What a great phrase - I'm going to steal that.

I hear you on the ISE. Honestly our goal was to do better than Notepad to help with adoption and then let the 3rd party ISVs deal with the heavy lifting so that we can focus our investments in the language and cmdlets. The thinking is that WE are the ONLY people that can improve the language whereas anyone can provide a better ISE.

There are quite a few good ones out there you try - PowerGUI (it has an ISE as well), PowerShell Plus, and Sapien. All of these have intellisense and are reasonably priced.

I like to say that with PowerShell, we are digging ourselves out of a 30 year hole which means that there is so much work to do on the language & cmdlets that we'll continue to focus on those and make incremental improvements in ISE. (Those might be really good increments but at the end of the day - they won't be our primary focus). Ultimately - the best experience comes when you think about what you want and are able to just type it because there is a commandlet to do it. In such a world, intellisense is not required. Each release, we strive to get closer and closer to that world.

BTW - I totally agree on your comments re: sys admins working together and getting better pay. One of my core goals with PowerShell is to give admins a tool that they can increase their pay with (because they become much more productive). If that doesn't happen - I won't feel successful. The good news is that I'm starting to hear some really good stories along these lines.

If you decide to continue on with PowerShell, I would again appreciate you blogging your experiences and pointing out where we can do better in the next version.

Thanks.

Jeffrey Snover [MSFT]
Distinguished Engineer
Visit the Windows PowerShell Team blog at: http://blogs.msdn.com/PowerShell
Visit the Windows PowerShell ScriptCenter at: http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx

Anonymous on Tuesday, December 28, 2010

Luke

One thing you will learn (which is pretty darn cool) that the very Architect behind Windows Powershell looks for the negative to be stated to improve it.

Powershell for me (as an Administrator of a Network) supplies options we didn't have before. Simple Command line based options for day to day tasks. Like Unlocking a user

Having said that, you have stated options that don't always meet everyone's day to day needs.

What language, console, interface was perfect day 1? None. Not a one.

Even BASH was grown over years of tweaking and support.

It pains me to see someone find a fault with Powershell, but just as EQUALLY I found joy in that someone was willing to share their PAIN in the system so that it can be improved.

I love Powershell but I also love to see how it affects and COULD improve others. You have shown where it needs improvement to others.

Rock on !

Sean
the Energized Tech

Comment by Luke Puplett on Tuesday, December 28, 2010

Hi Sean, thanks very much for commenting. My book arrived and I'm trying to fully learn PowerShell so I can make an informed opinion.

I'm finding some simple things a bit tough. It seems the conventions required to tame what is complex, complicate what is simple. It's a trade-off.

Mostly though, my feeling today is that PowerShell exposes the .NET Framework but scrimps on the rich assistive technology that I have at my disposal because of the great work that Scott Guthrie, Rico Mariani, Jason Zander et al are doing in Visual Studio.

I understand and agree with the philosophy that Jeffrey Snover alludes to, that PowerShell is a democracy and 3rd parties can and are stepping-up and delivering great ISEs, but its not without its risks.

I wholeheartedly agree with your sentiments regarding Jeffrey taking the time to monitor blogs for feedback, as well as taking the time to write.

With all the evangelism that Microsoft are doing, it's important to tune out the cheering squad and keep an ear firmly on the ground.

Ugly as the saying is, its true that the game is over as soon as you start believing your own BS.

That's a topic for a blog post in itself; "The Evangelist's New Clothes" 8¬)

Luke

Comment by jsnover on Monday, January 03, 2011

>It seems the conventions required to tame what is complex, complicate what is simple. It's a trade-off.

It's true. Here is the way I think about it. At the end of the day, simplicity comes from having a cmdlet that does what you want it to so that you can just THINK/TYPE/GET what you want. We made a conscious decision to invest in those things that would incent other teams to produce those Cmdlets and the ramification of that is that we had less to investing in making some of our stuff simpler. We'll fix that over time (the surface area is enormous so it won't happen overnight).

We now see every major product ship with PowerShell support and every product that ships with PowerShell finds the experience and customer reception so good that they all plan additional investments in future releases. This makes me believe that we made the right tradeoff but it doesn't lessen our need to go back and make things simpler.

> Ugly as the saying is, it's true that the game is over as soon as you start believing your own BS.

Absolutely correct!

That is one of the things I learned from BillG. We would go solve all the problems that he pointed out and in the next review, he would tear us to shreds. At first I thought, "what an ungrateful guy". Later I understood - Bill has the gift of dissatisfaction. He always pushes us to go farther faster no matter how much we had already done.


Enjoy!

Jeffrey Snover [MSFT]
Distinguished Engineer
Visit the Windows PowerShell Team blog at: http://blogs.msdn.com/PowerShell
Visit the Windows PowerShell ScriptCenter at: http://www.microsoft.com/technet/scriptcenter/hubs/msh.mspx

Comment by RashRL on Thursday, February 16, 2012

I have been looking for posts of others having issues with Powershell, ane not having much luck. Mostly seems like evangelizing. I have been writing vbScript since it came out and have written some very complex AD monitoring/reporting type scripts, never having resource issues in doing so. I really want to like PowerShell but am not having much luck at the moment. I wrote a simple script to query AD for users who had proxyAddresses, storing them in a hash table and then doing a little work on them. Output to a logfile is minor. The script took 19 minutes to run, and consumed 3GB of memory. I wrote it over in vbScript and it took 9 minutes and consumed 30MB of memory. The other issue is running a script in a loop for an extended time frame. The vbScript works fine an never builds up beyond the initial 30MB of memory consumption. If I put the PowerShell script into a continuous loop, it adds 3GB of memory each pass, never giving any up. I have tried invoking the garbage collector and disposing of the AD searcher with no luck. There are times/scripts that these issues would not bother me and I am going to continue learning/advancing my PowerShell skills. Your idea of learning C# is appealing. I would have done that a long time ago, but most of the accounts I worked on would not allow compile programs (written by admins) to be used in their production environment. That has changed, so...... Anyway, just dumping a little here but maybe someone else has run into these issues and knows something I may be missing.

Randy Rash
HP Tech. Consultant IV
randy.rash@hp.com

Post a Comment