upgrade from SDK 2017R3 to SDK20xx Ry Fails

MIP
, ,
1

We hit an issue while upgrading our application from MIP SDK 2017R3 to a newest MIP SDK version (MIP SDK 2018R3 and MIPS SDK 2020R2 versions have been tested).

A word about our application context:

Our application is composed of plug-ins consumed by a host application (SCADA type application).

Our plug-ins are .Net 4.5.2 based and use the MIP SDK component integration (ImageViewerControl) to provide video features inside the host application.

The host application is a native Winform application.

While using the MIP SDK 2017R3 everything works fine and video streams are correctly displayed.

However, to benefit from hardware acceleration, we had to switch to a newer MIP SDK version, and following the upgrade our video streams ceased to be displayed.

To migrate to a new SDK we had to switch our plug-ins to .Net 4.6 (required on some newer SDK version!) and we basically just installed the new SDK on the target machine and upgraded our post build event to use the new SDK provided batch files (copyUIFiles.bat). Nothing else was modified from our side!

Using MIP SDK 2018R3, we tracked down the issue to a missing file error related to “WPF” while calling Initialize() method on the ImageViewerControl:

System.Windows.Markup.XamlParseException: Could not load file or assembly ‘VideoOS.UI.Common.WPF, PublicKeyToken=bc60fba4a7969f89’ or one of its dependencies. The system cannot find the file specified. —> System.IO.FileNotFoundException: Could not load file or assembly ‘VideoOS.UI.Common.WPF, PublicKeyToken=bc60fba4a7969f89’ or one of its dependencies. The system cannot find the file specified.

at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

at System.Reflection.Assembly.Load(AssemblyName assemblyRef)

at System.Windows.Baml2006.Baml2006SchemaContext.ResolveAssembly(BamlAssembly bamlAssembly)

at System.Windows.Baml2006.Baml2006SchemaContext.ResolveBamlTypeToType(BamlType bamlType)

at System.Windows.Baml2006.Baml2006SchemaContext.ResolveBamlType(BamlType bamlType, Int16 typeId)

at System.Windows.Baml2006.Baml2006SchemaContext.GetXamlType(Int16 typeId)

at System.Windows.Baml2006.Baml2006SchemaContext.GetPropertyDeclaringType(Int16 propertyId)

at System.Windows.Baml2006.Baml2006Reader.GetStaticExtensionValue(Int16 valueId, Type& memberType, Object& providedValue)

at System.Windows.Baml2006.Baml2006Reader.Process_PropertyWithExtension()

at System.Windows.Baml2006.Baml2006Reader.Process_OneBamlRecord()

at System.Windows.Baml2006.Baml2006Reader.Process_BamlRecords()

at System.Windows.Baml2006.Baml2006Reader.Read()

at System.Windows.Markup.WpfXamlLoader.TransformNodes(XamlReader xamlReader, XamlObjectWriter xamlWriter, Boolean onlyLoadOneNode, Boolean skipJournaledProperties, Boolean shouldPassLineNumberInfo, IXamlLineInfo xamlLineInfo, IXamlLineInfoConsumer xamlLineInfoConsumer, XamlContextStack`1 stack, IStyleConnector styleConnector)

at System.Windows.Markup.WpfXamlLoader.Load(XamlReader xamlReader, IXamlObjectWriterFactory writerFactory, Boolean skipJournaledProperties, Object rootObject, XamlObjectWriterSettings settings, Uri baseUri)

— End of inner exception stack trace —

at System.Windows.Markup.XamlReader.RewrapException(Exception e, IXamlLineInfo lineInfo, Uri baseUri)

at System.Windows.Markup.WpfXamlLoader.Load(XamlReader xamlReader, IXamlObjectWriterFactory writerFactory, Boolean skipJournaledProperties, Object rootObject, XamlObjectWriterSettings settings, Uri baseUri)

at System.Windows.Markup.WpfXamlLoader.LoadBaml(XamlReader xamlReader, Boolean skipJournaledProperties, Object rootObject, XamlAccessLevel accessLevel, Uri baseUri)

at System.Windows.Markup.XamlReader.LoadBaml(Stream stream, ParserContext parserContext, Object parent, Boolean closeStream)

at VideoOS.Platform.SDK.UI.DirectX.SmartClientVideoPlayer..ctor()

at VideoOS.Platform.SDK.UI.DirectX.ImageViewerDirectX..ctor()

at VideoOS.Platform.SDK.UI.SDKImageViewerControl.Create()

at VideoOS.Platform.SDK.PlatformImageViewerControl.Initialize()

However, the missing file is present in the target folder, so we guess one of its dependencies is not available, but which one? Using dependency walker (Depends.exe) on this did not help…

While using MIP SDK 2020R2, we tracked down the issue to another missing file error related to “toolkit*.dll” while calling Initialize() method on the ImageViewerControl:

VideoOS.Toolkit.FileException: Could not open file ‘ToolkitRendererPlugin_vr.dll or CoreToolkits.dll’!

at VideoOS.Toolkit.ToolkitFactory.CreateInstance(String configurationXml)

at VideoOS.UI.Common.WPF.VideoPlayer.Toolkit.ToolkitWrapper..ctor()

at VideoOS.UI.Common.WPF.VideoPlayer.Toolkit.VideoImage.VideoImageManager..ctor(Dispatcher dispatcher)

at VideoOS.UI.Common.WPF.VideoPlayer.Toolkit.VideoImage.VideoImageManager.GetVideoImageManager(Dispatcher dispatcher)

at VideoOS.UI.Common.WPF.VideoPlayer.Toolkit.VideoImage.VideoImage..ctor()

at VideoOS.UI.Common.WPF.VideoPlayer.Toolkit.ToolkitVideoSource..ctor(MainModel mainModel)

at VideoOS.UI.Common.WPF.VideoPlayer.VideoPlayer..ctor()

at VideoOS.Platform.SDK.UI.DirectX.SmartClientVideoPlayer.Initialize()

at VideoOS.Platform.SDK.UI.DirectX.ImageViewerDirectX.Initialize()

at VideoOS.Platform.SDK.UI.SDKImageViewerControl.Initialize()

at VideoOS.Platform.SDK.PlatformImageViewerControl.Initialize()

CoreToolkits .dll is there but we could not find any ToolkitRendererPlugin_vr.dll anywhere including inside Milestone folder. Is this file really missing ?

What did we do wrong during our “SDK upgrades” that leads to the same issue (no more video displayed) ?

Once again the whole application is working fine using SDK 2017R3, what kind of changes were made to newer SDKs that could lead to this kind of issues ?

Any help would be much apreciated, we’ve been stuck on this for many monthes now.

2

Can you please update SDK to 2020 R3 which is newest and requires to have .NET 4.7 -

https://doc.developer.milestonesys.com/html/index.html?base=gettingstarted/plug-in_development.html&tree=tree_1.html

Please make sure to get rid of old files, otherwise SDK wouldn’t work properly. So please not to have mixed files old and new.

3

We did also tried SDK 2020R3 after switching our modules to .Net4.7 without any success. We collected the very same error messages we had using SDK 2020R2.

Regarding the link you kindly provided, as I mentioned we created Plugins for a SCADA application. We are not developing plugins for XProtect clients at all, and we are not using the Plugin integration part of the SDK.

Is there any hint or explanation regarding the error messages we are receiving?

4

We really think this must be a mix of dependent dlls.

Please double-check that you remove all dependent dlls from the older MIP SDK and then copy new files from the new MIP SDK (bin folder) and rebuild. We have seen very often that exception message like the one you mention is caused by having a mix of dlls.

5

We double checked and actually restarted our tests from scratch making sure that no SDK files could be mixed up.

We obtained the same results.

Using depends.exe from the SDK (not the version from dependencywalker.com we previously used) on the file “VideoOs.UI.Common.WPF.dll” from the SDK Bin folder, we obtain the results from the attached picture.

Milestone Depends
Milestone Depends500×480 66.7 KB
None of the missing files were found in the SDK bin folder (SDK2020R3) BUT some files could be found into the Smartclient folder only.

Manually copying these files to our target directory (the SDK .bat file does not copy them)… did not improve the final result. Our Application running in the native host reports the same error.

I think it is worth adding that our solution simultaneously build (from the very same SDK files source) another managed executable (.Net exe), that consume the very same plugins we created for the native host application.

This managed executable always works no matter what SDK version is used:

No missing files or missing dependencies is reported by this managed application.

The managed executable works perfectly while it runs alongside the native application although the latest is not able to display any video after switching both hosts, to the same newer SDK.

The attached table picture attempts to summarize the issue while simultaneously running both executable on the same computer.

summary SDkWould this issue be possibly related to some changes made by Milestone to the ImageViewerControl to better support WPF in SDKs following 2017R3 ?

Note: I previously mentioned our native host is a winform only application and our managed host is also a winform only application.

6

Seen in other cases: It might be that Environment.CurrentDirectory does not resolve to the folder where the MIP SDK components are located. If this is the case you will get errors indicating missing stuff. Can you add logging where in your code the exceptions are raised?

7

We can’t say for certain why the files are not copied correctly and you are missing dll’s. We expect their may be some issues with the CopyUIFiles.bat or placement of files.

But instead of trying to figure out exactly what is wrong, we suggest that you use our nuget package in your projects. You can find our nuget packages by searching for Milestonesystem in your nuget package manager. Select the MilestoneSystems.Platform.Videos.SDK package

If you havn’t used nuget before, here is a short description about what to do:

  1. Rightclick on your solution or project, select “manage nuget packages”
  2. In the top of the newly opened window, ensure “browse” is selected
  3. Search for Milestone systems
  4. Select MilestoneSystems.Platform.Videos.SDK and then click on install
  5. This will install the package and once you build your solutions, all relevant files will be added
8

We’ve had additional talks in the team and have two more questions. We think you should focus on 2020 R2 or R3 (or Nuget). Could you send us a screenshot of MDepends on the “CoreToolkits.dll”, to see details of any dll’s missing?

Also we would like to know if you are running 32 or 64 bit? This could make a difference when we investegate which installer has the problem.

And a side note, the “ToolkitRendererPlugin_vr.dl” is an old dll, which is not used anymore. Issues with this Dll, should be ignored

9

We agree with you, and we are now focussing on making this work with SDK2020R3 (not the nugget version, at least for now).

@Bo Ellegård Andersen (Milestone Systems)​ The Environment.CurrentDirectory property point to the directory where the native host is installed. This directory is the same when we use SDK2017R3 or SDK2020R3.

@Henrik Hein​ Please find attached a screenshot from Mdepend on the “CoreToolkits.dll” from the SDK bin directory.

mdepend_coretoolkit
mdepend_coretoolkit233×478 38 KB
It doesn’t seem to miss any dependencies whereas the error message we receive seems to be contradictory.

The native host is a 64bits process running in a 64bits OS.

Our .Net assemblies using the SDK are .dll compiled as “Any CPU” (and have been working “as is” until our SDK migration attempt)

We made extensive tests with SDK2020R3, results will follow.

10

We started over and made extensive tests with SDK2020R3, here are our results:

First, before switching to SDK 2020R3, we upgraded our plugins from .Net4.6 to .Net4.7.

We then successfully validated that both Native host and Managed host were still working normally using SDK 2017R3 + .Net 4.7.

We switched the build process to use SDK 2020R3:

No changes with the Native host, same missing file error returned and no video displayed at runtime.

The Managed host started to act:

  • unable to display streams coming from some physical IP cameras: ImageViewerControl shows the “Connected to ” on a black background while diagnostic overlays show rendering counter overflow, off the charts (We guess some stream packets seem to be received but are not displayed ?).
  • Able to display streams coming from StableFPS

Then, following Bo’s hint about path, we changed the Native host shortcut property “Start in” to point into our deployment directory where SDK files are deployed – Please take note, this is not the manufacturer’s recommended setting to launch this host’s applications –

Following this change, the missing file error disappeared, and the native host then started to behave just like the Managed host:

  • unable to display streams coming from some physical IP cameras: ImageViewerControl shows the “Connected to ” on a black background (rendering overflow counter increasing fast).
  • Able to display streams coming from StableFPS.

We noticed that SDK2020R3 encounters the “no video displayed” issues with some camera configuration that were working with previous SDKs.

For example, with some camera brand only, when Xprotect camera streams configuration is set to use stream 2 and 3 only from the camera (leaving out stream 1), SDK 2020R3 can’t display stream. While SDK 2017R3 can successfully display video using this very same configuration.

Switching camera configuration to use Stream 1 and 2 only (leaving out stream 3) enables SDK 2020R3 to display video in both native and managed hosts.

We noticed that SDK2020R3 (and more generally SDK versions following SDK2017R3) introduced subtle and undocumented changes in behavior and deployment that make it kinda difficult (and unfortunately costly!) to upgrade from a previous SDK version.

Here is a non exhaustive list of what we found out so far:

  • SDK files deployment does not seem to be transparent anymore, some dependencies issues were introduced (see the change we had to apply to our native host shortcut).

  • Some camera stream configuration applied to some camera brand, are causing new and unexpected behavior when trying to display streams from these cameras.

  • Some useful features like diagnostic overlays do not work the same now: Simply setting up diagnostic mode once at application startup does not work anymore for us, it required some code changes to be enabled while a video is already displayed.

Have you already identified similar issues ?

Are there plans to bring back SDK to well known behaviors, particularly the dependency behavior, which is very restrictive for us ?

11

Some of description of the issue seems very much like a camera loosing connection with the recording server. Please always double-check if the Smart Client can use the camera footage or fails the same way.

The upgrade of the MIP SDK and underlying ImageViewerControl means that you will now utilize hardware acceleration, it is important to make sure your graphics drivers are up to date. We have seen many issues that are solved simply by updating the graphics cards drivers. On these issues we expect the issue to be the same using the Smart Client (same version) so please test this in the same PC.

I was not aware the diagnostics overlay behaves differently, I will take your word for it and I am glad you have been able to do code changes to get this working.

The MIP SDK will not be changed back, hopefully we will be able to help on finding a solution.

12

Regarding the SDK files deployment part we made an experiment where we made a small application that did nothing but calling some methods on a dll using reflection. The dll was in a separate folder along with all the dll’s from the SDK and provided an ImageViewerControl.

The application folder contained nothing but the application itself and thus no dependencies could be taken from there. The ImageViewerControl was returned fine by the dll and was able to play back video using hardware accelleration.

So it seems that all the appropriate dependencies are available and nothing in the SDK requires them to be in the current directory of the executing application.