Virtualisointi EQU virtualization

Now that 4.6 is officially out, many of you who are existing Softgrid / App-V customers are looking into migrating to this new version, especially if having Windows 7 projects etc. going on as well.

However, what Microsoft recommends you to do in such case is to upgrade all your virtual application packages into new 4.6 format. This was also true and official recommendation when 4.5 came out back in the day. But if you really want to do this for all of your packages, opening up every package one-by-one with 4.6 Sequencer, and then saving out (remember, you need not do anything besides Open + Save, this effectively writes out SFT file in new format and creates manifest file if none existed before) can be tedious task.

Also somewhat depending on what App-V infrastructure model is used, there might be need to change streaming URL (HREF attribute in the OSD file’s CODEBASE element), set of supported OS values and perhaps other aspects for the package,

Fortunately there is a better, not to mention much quicker, way to do this tedious task with little actual human involvement. The answer of course is my SFT Encoder product, which is designed to be used as command-line based App-V package processing utility and with batch processing GUI, handy mass-migration tool. As 2.0 version now fully supports new App-V 4.6 package format in both 32-bit and 64-bit environments, this means that you can migrate your old packages to newest format with automated process in one go!

As a brief example, let’s look at how to upgrade batch of packages to 4.6 ready versions:

After installing SFT Encoder, we run it once from the command-line and supply license with /license -switch (note that this is not needed with trial version). After that, we can start using batch processing GUI.

Firstly we have to tell where from source packages are read in, by supplying directory path under from Encoder scans for available packages. We also specify where to packges are to be output when conversion has been done: 

 Next, we specify parameters for conversion. In this case, we want to change fileformat version (as selectable from Version -dropdown list) but even if it is technically possible to change block size setting to match 4.6 default (and for new packages, only) of 64 kB it’s generally a ad practise to change it afterwards and not really supported by Microsoft either. We also want to put outputted packages each to its own subdirectory on target directory and we want to change streaming source for SFT file in all of the OSD files to match our new target environment.

As a special treat, we also want to remove possible Windows Installer runtime files found inside the VFS as part of the processing. As you might remember, App-V up until recently defaulted into adding those runtime files for all new packages unless explicitly removed during sequencing wizard. Problem with MSI runtime files is the fact that they may prevent package from self-repairing virtual environment on the client for the current user if Windows Installer runtimes differ between host and DLLs stored in the VFS.

If we wanted, we could have prevented generation of new history entries or even removing all traces of past history entries, but in normal migrations there is really no need for it. If however deciding to so, these two settings cannot be mixed together as that would result invalid SFT file!

Lastly, we can adjust other few content generation/modification aspects for processed packages, if for instance we wanted to inject some registry or file data into the package along the way. In this case, we are only intrested in removing all possible OS -elements from the OSD files so that they will show in our brand-new App-V client environments, be it 32- or 64-bit or workstation or RDS session host: 

 When we are ready to proceed with actual conversion process, just hit Start Processing -button!

Batch processing GUI will then go through all subdirectories in the selected source directory and tries to detect all Softgrid/App-V packages from the presence of SPRJ files, and then issues appropriate parameters to actual SFT Encoder executable (command-line -based, but run hidden) to process each package individually: 

When all is said and done, we have collection of App-V 4.6 conforming packages in our output directory and beneath, without having to go and processing each with Sequencer one-by-one. Of course this does not imply that packages shouldn’t be then tested with actual clients, since sometimes some old packages might refuse to work in the new environment especially if operating system changes underneath. But at least with larger number of packages time saving can be significant, especially since you can also automate annoying server address and OS changes within OSD files.

Please note that SFT Encoder will not output MSI wrapper installer (at least not yet), but all other package files including the manifest are automatically generated if none exists in the source package. And there’s quite much more than just this for SFT Encoder, since batch processing GUI exposes only few of the options available, those most common things needed during mass-migration.

If you don’t want to jump in and purchase SFT Encoder right away (which you should of course :-)), there is a trial version available for time-limited testing.

Happy conversions!

Microsoft has just announced public availability of official App-V 4.6 version.

The highly anticipated 64-bit support is now here as are few other enhancements plus two new App-V Resource Kit tools, Application Virtualization SFT View Tool and Application Virtualization MSI Compat Transform.

Both Desktop (part of MDOP 2010 package) and RDS versions of App-V 4.6 are immediately available.

One of the admittedly greatest puzzles in Microsoft App-V usage has so far been the lack of visibility for the PKG files.

These PKG files – as all App-V admins very well know - basically contains file and registry data that has been cached for the user and for the machine after App-V enabled application has run on the client. To make things more complicated, it is just not user and machine data as such (like Sequencer presents the division with Application Data and User Data in UI) but it also involves the factor of who has made change in the client system. And “who”, in this context, being either present user or system itself. Before Softgrid 4.2 really, this was more clear-cut and simple: changes to files marked as Application Data/Config were visible to all users on the client machine whereas files marked as User Data/Config were visible only to user who initiated the change. But alas, not anymore.

On a physical level, App-V client caches data to several different PKG files depending on who’s change we are talking about:

• UsrVol_sftfs_v1.pkg file in user’s own (roaming) Application data represents changes to files marked as User Data that are initiated by and visible to processes of that user (i.e. are running as that user), usually meaning applications you  (as user) launch and use from App-V package. This file also contains cached virtual environment configuration, which practically means changed virtual registry, for the user.
• UsrVol_sftfs_v1.pkg file in per-machine App-V cache (C:\ProgramData\Microsoft\Application Virtualization Client\SoftGrid Client\AppFS Storage in Windows Vista and 7, under All Users -profile in earlier Windows versions) represents changes to files marked as User Data that are initiated by and are visible only to few selected Windows system processes (like winlogon.exe) having access to Q: drive.
• GlblVol_sftfs_v1_user_SID.pkg file in per-machine App-V cache represents changes to files marked as Application Data that are initiated by and are visible to processes of user account having SID in PKG file’s name. Normally, you should see one with your account’s SID in there but on machines having multiple users there’s one for each.
• And finally, we have GlblVol_sftfs_v1_S-1-5-20.pkg file in per-machine App-V cache representing changes to files marked as Application Data that are initiated by and are visible to system level processes running in the virtual environment. These processes are, for instance, virtual services started from the package. This file also contains cached virtual environment configuration for the system.

 So knowing that there’s multiple contexts involved for caching, how do we know what the heck is actually cached (as opposed to coming from original SFT) on the client since sometimes this is needed for troubleshooting purposes? Sad answer is that we actually do not know this, since App-V Client does not provide any tool to view cached information besides starting cmd.exe or such in the package’s VE and trying to poke around in system with registry editor etc. The problem of course is that this method does not provide any information which of the files or registry values differ from what we have from SFT itself as everything is merged down for us by App-V itself. Also, this method does not reveal contents of alternative contexts, since we (or our processes) are by definition operating under our own user account.

Thus, the holy quest for PKG viewer :-) Up until now, nobody has been able to produce such viewer (at least publicly!) and Microsoft has not been very forthcoming in opening up PKG fileformat (which incidentally is actually the same fileformat as App-V’s very own package cache, sftfs.fsd file, uses). But, in preparing for very near release of SFT Explorer, version 2.0, I decided to once and all to resolve this need and am happy to report that I have been able to do it!

Code is not finalized yet, but I know how to do it, and as a proof here’s four different view of cached [PKG] contents of Firefox 3.0 package that I have been running for ages on my machine:

Here’s how the UsrVol… -file looks like for the current user:

Here’s how the UsrVol… -file looks like for the system:

Here’s how the GblVol… looks like for the user:

And here’s how GblVol… looks like for the system:

As you can see, there’s clearly a vast difference how different processes see the world, and these screenshots are not even including virtual registry!

–

As stated above, this PKG viewing capability is now likely to be part of upcoming version of SFT Explorer, and to track things related to it I invite all to join to SFT Explorer Users -group in LinkedIn.

As a natural succession to a fact that Microsoft has now included App-V’s Terminal Server (Remote Desktop Services) version usage rights in its RDS CAL (previously known as TS CAL), this version is now downloadable straight from the Microsoft Download site instead of using Licensing -site as before.

However, to download this package, you first have to enter Product Identification Key of Windows Server 2008 or 2008 R2 TS into mandatory registration form..

I noticed that Remote Desktop Services Team has posted a new App-V related script to Microsoft TechNet Script Center.

This script is mainly meant for running locally installed apps inside App-V’s virtual environment – without sequencing them – in order to quickly fix certain compatibility problems that App-V’s VE can make go away (like, for instance,  registry change conflicts). This in no way  means that application being run this way is truly virtualized application and that it couldn’t in any way conflict with other locally present applications.

To do its “magic”, script utilizes one lesser known feature in App-V Client, first introduced in 4.5 version, making it possible to specify alternative executable which is launched instead of file pointed by OSD’s FILENAME attribute.

It should be noted that the script requires that Default App-V Application (DefaultApp.osd) has been published to a client as script uses that OSD as virtual environment for application.

In this multi-part series, we take a look at under covers of files that make up an App-V package. Some of these files are in binary format, which makes them interesting more or less to those of us who develop software, while those files that are human-readable (XML formatted) textual data can easily be examined and edited by App-V administrators and packagers using normal text editors and tools.

 

Overview of package files

We begin our journey to App-V files by taking an overview on what files (by extensions) you generally can expect to see when looking at the directory holding a complete App-V package:

.SPRJ file

This file, of which there is only one per each package, is Softricity Project File, binding logically all other package’s files together. This file is used by the Sequencer and optionally Management Console which is associated with Management Server –based infrastructure.

.OSD files

Per each package, there could be a multitude of these files, which derives its name from ages old (and mostly forgotten) RFC note called The Open Software Description Format (http://www.w3.org/TR/NOTE-OSD.html), originating from Microsoft and company no longer with us, called Marimba. This file used and required by the Sequencer and App-V Client, each OSD file representing one single startable application from the package. Or as another way to look at it, each one representing one particular shortcut you intend to publish to client machines.

.SFT file

This file which got its name from Softricity, the company who originally created App-V, nee SoftGrid, is the second absolutely required file by the App-V Client. It contains all the actual content (applications and related files plus virtual environment) of the App-V package, packaged in binary format somewhat like in ZIP archive.

.ICO files

In one package, there could be number of standard Windows icon files, which map either to application shortcuts (i.e. OSD files) or to file types associated with individual applications found inside the package. Since App-V packages do not carry executable (EXE) files for applications outside the virtual environment, Windows needs to have icon files separately so that users can see familiar program and document icons when virtual applications has been deployed to their machines. For this reason, Sequencer extracts these icons as “naked” ICO files to the package directory.

.XML file

Each package has one package publishing -related file, called Manifest file. This file contains aggregated information from SPRJ file and OSD files, and it is used (quite rarely directly) by App-V Client but more prominently so by Microsoft System Center Configuration Manager (SCCM) to make the whole App-V package and all individual applications from it available to the client, when no App-V Management Server is available as source for publishing information.

.MSI file

As an optional and additional file, standard Windows Installer database, MSI, is created by the Sequencer, one per each package. This MSI can be used to deliver the package to App-V Clients as alternative way to using App-V Management Server publishing or SCCM delivery.

 

In addition to these physical files that you can observe directly in the package directory, each SFT file holds – among other application related files – one App-V package specific file which is also referenced from each and every OSD file in the package:

.CP file

This special file, having extension officially coming from “Computer Protection” (thanks for Tim for revealing that bit of info ;-)), is the virtual environment’s configuration file for the package. It contains not only the virtual registry in it, but also something called Virtual Filesystem (VFS), and it is used by the App-V Client while establishing (or “uploading”) Virtual Environment when virtual application is launched.

 

---

In the upcoming articles of this series, we will take a closer look at the contents of following file types described above: SFT, CP, OSD, SPRJ and XML.

Stay tuned!

Sorry, this entry is only available in Suomi.

Sorry, this entry is only available in Suomi.

One of the more obscure functionalities in App-V is the concept of differential SFT files. These DSFT files were introduced into product during 4.5 Release Candidate -phase, at which point you actually had UI option in the Sequencer for generating such an output:

 

In a very surprising move, this option was actually removed from the Sequencer when RTM was released. I guess it was confusing enough that they felt better to hide the whole thing rather than trying to explain it.. But fear not, DSFTs are still with us (yes, even in 4.6 beta), it’s just that you don’t really see it anywhere. The purpose of this post is to explain them and also show how you can use DSFTs and why.

 

So what is DSFT anyways?

Differential SFT files are, like the name implies, SFT files that contain the differential data (delta) between two versions of the SFT files; the older one and the upgraded one. This is a bit like what you would expect from the MSP files in context of Windows Installer, but of course it is not fully analogous.

DSFT files contain the identical headers to the upgraded SFT file, but only those blocks (FB 1 and 2) that is new to that particular version. This basically enables you to deliver upgraded App-V packages just as that, as upgrade, instead of traditional way of delivering new full SFT file which may contain loads of [old] data the client already has. With streaming, this is of course no problem because of differential streaming, but when servicing standalone client it would be handy to deliver only those bits that matter.

I should stress that the differential SFT file is just the differential between two consecutive versions i.e. if you have package in the client cache that is from the older SFT version (not previous to the most current one) you are not able to bring it up-to-date with the latest DSFT file; you would actually need to have one DSFT file applied per each upgrade version!

 

How do I create one?

As previously said, there used to be option in the Sequencer for a very brief while for creating a DSFT file. You can notice still in 4.5 builds the strange missing line in the checkboxes for Deployment -tab, that’s the placeholder where the DSFT checkbox used to be.

Now without GUI based option, out only choice – at least for time being – is to use executable that actually ships with Sequencer but probably not many knows about: mkdiffpkg.exe.

This tool is a very simple command-line based DSFT generator, which basically needs one input file and one output file; SFT file and DSFT file, respectively. The whole idea is that you choose your upgraded SFT file as source and the tool scans what blocks match the SFTs version (remember, as of 4.5, App-V Sequencer assigns file versions to match the SFTs version, instead of incrementing by one by each change as it used to be). These blocks are then written out as DSFT file, specified as second argument for mkdiffpkg.exe, ready to be delivered.

One of the reasons the checkbox in Sequencer UI was confusing is exactly this; DSFT generating is clearly an post-Sequencing task which did not sit in the flow how Sequencer is used. If you enable the check-box, does that mean you want changes for this version or the one you trigger by saving? You cannot also create DSFT against your very first version of SFT, that would not make sense. And hence the checkbox was always grayed out when doing Sequencing (adding to the confusion still), only when you opened package for upgrade you could enable it.

DSFT creation example

So, let’s create an example difference file for one of the applications available at my InstantApp -site. For simplicity’s sake I’ll choose VirtualDub which is an AVI video editor and provided in ZIP file instead of full-blown installer. The version I’ll base my upgrade on (and which is available at the site in time of writing this) is 1.6.14, clearly not the latest anymore as one available from VirtualDub’s SourceForge site is 1.9.5.

As a first thing to do, I’ll perform an usual upgrade to App-V package. Note that my package was originally created in 4.0 (!) Sequencer, so the upgraded one is by necessity now brought to 4.5 level (I’ll use 4.5 RTM for this):

 

 

Ok, so for this upgrade I simplycopy few files over from the extracted ZIP file to my package’s directory structure on Q: -drive:

 

Then we start the main application (VirtualDub.exe) like you always should (2-3 times) and this time for good measure; it asked about license agreement and also presented first-run dialog. Moving on and after stopping the monitoring, I’ll remove unnecessary MSI runtime files and also modify shortcut to reflect the changed version number.
After not optimizing for FB1/2 (it’s too small package to make any difference with such a thing) I’ll examine the results before saving and see that some of the files are not at version 2 but still at version 1. These are the ones that do not get included inside DSFT file I’m going to generate in a short while:

 

 

Next I’ll just simply hit save in order to create v2 version of my VirtualDub App-V package and sure enough, I soon have my package files for version 2:

 

Comparing this new version of SFT file, I see that it’s size has increased by over 2 megabytes over v1 of the SFT file. Now only thing left is the interesting part, the DSFT generation:

 

Yes, it’s that easy ;-)

Looking at the DSFT file generated, it’s 3388 kB in size, so we really couldn’t save much space (3419 kB – 3388 kB = 31 kB!) in this particular upgrade scenario, but for packages where less files changes the saving could be significant. Just think of Office hotfix for instance..

 

How/where can I use this DSFT file?

That’s a very good question. As it is, there’s no UI options or indication in the MSI wrapper or SCCM integration or, well, anything anywhere that would suggest an ability to use DSFT files. But that does not mean there actually isn’t way to utilize those delta files!

Yes, I’m talking about good old sfttray.exe here. This is the executable that MSI wrapper also uses in the background for loading (importing) SFT file when you deliver virtual applications via MSI file to the client. And it is that very same syntax you can use for reading in DSFTs, just simply by pointing to DSFT file instad of SFT:

sfttray /load “MyApp 1.1″ /SFTFILE c:\tmp\MyApp_2.dsft

This will trigger loading of package into the client’s cache, but instead of loading the whole package we actually load just the bits available inside DSFT, resulting an update to the package without overhead of moving whole SFT around.

As can be seen, this would be very easy to incorporate into one’s own scripts. Or, maybe even to use when delivering those wrapped MSI packages, but instead of pointing to SFT just point to DSFT!

DSFT usage example

Continuing our earlier example of VirtualDub, let’s now apply the difference to the client. For this purpose, I’ll have 4.5 client installed which has already received the v1 of the VirtualDub package from Management Server. My task is to apply v2 version to the client using DSFT version instead of publishing our upgraded v2 SFT through Management Server:

 

 When launching VirtualDub at the client, I can verify from the GUI that it’s indeed original – unupgraded – one:

 

So let’s do an upgrade via DSFT file we generated after sequencing.
Remember to close all applications from the package, otherwise you cannot upgrade package data in the cache (DSFT or otherwise)! You also need to have permissions set to allow importing or be local admin in order to import via sfttray and you HAVE to specify full path to the DSFT file:

 

 

 Now, only thing left to do is to check if our upgrade with DSFT actually succeeded to achieve what we wanted. As we could see the version directly from the VirtualDub GUI, simply re-start it from the existing shortcut and see what happens..

 

Success! :-) We managed to upgrade our package with DSFT file!

 

Although there is one caveat you should be aware of (and I bumped into this during my testing): if your original package has been delivered to desktop via Management/Streaming Server (as mine had), client will try to do downgrade of the package from DSFT upgraded one when you launch it, unless you have matching updated SFT file published in the centralized server as well. The reasonfor this is very simple: client will have a check with the server (if it can be reached) each time you launch an virtual application and if the server contains older SFT than one you delivered with DSFT, client will think it needs to revert back to older version (”Active Downgrade”, if you will)!

In any case, DSFT is something you definitely can take advantage of by following the advise I have outline in this article. Just be careful with it when combined with the streaming delivery.

–

You can download my upgraded package and DSFT file used in the examples from here and here.

Happy DSFT’ing!

As the upcoming App-V 4.6 now supports x64 systems (along with 64-bit apps) so new OSVALUEs were in order. These are the new values for x64 OSes (existing – old - ones are for 32-bit versions):

• WinXP64
• WinVista64
• Win764
• Win2003TS64
• Win2008TS64
• Win2008R2TS64

Speaking of OSVALUEs, it should be noted that no longer separate values exists for both “regular” server and terminal services. From now on, only one value maps for server OSes and that’s for terminal server (understandably so as not many people ran App-V client on regular servers and also when considering licensing nowadays, that’s not really even doable). For 2008 R2, only 64-bit value is defined.

-Kalle