Please let me trade

A case study in diagnosing failure.

May 16, 2023

tldr; I tried using Fidelity’s Active Trader Pro on macOS but it wouldn’t install. This is my path to discovering that the CrossOver’s utility program cxwget.exe was unable to negotiate a ssl connection with microsoft to download the .Net 4.6.2 runtime due to DigiCerts changes on March 8. And a subsequent fix using curl to get me trading again.

I got my first taste of trading in 5th grade (1993) when I was fortunate to be taught by Ms. Janet Posen who had an immense influence on me and my education. We would look up stocks in the newspaper, find some we thought would be good, and follow them over the course of months. I remember winning a few weeks and I made money overall. It was fun and exciting.

Options As A Strategic Investment opened my eyes to a whole new world and I have progressed past the newspaper days to leveraging things like koyfin and tikr and having accounts with scottrade, vanguard, fidelity, ibkr, webull, robinhood, etrade, TD, schwab, and probably a few others. I am fairly curious about a lot of things and tend to dive deep learning about new products, where they succeed, and where they fail.

I have recently found some more time on my hands and while I have had some success in trading over the years mostly buy and hold with some opportunistic shorter duration trades mixed in, decided I wanted to dive a bit deeper into the desktop trading platform Active Trader provided by Fidelity to see what it offers.

Let’s Get Going

First stop. Download the App. One google search away, easy download with direct links if you want to share the applications or script them in the future. Great! +1 for Fidelity.

I have run all sorts of desktop/server environments over the years and am comfortable in any OS. For my daily driver I currently am running macOS. Everything updated and latest.

Run the installer. It fails.

But it does leave the /Applications/Active Trader Pro.app application hanging around. And if I had been staring at my screen during the install I would notice that the Active Trader Pro.app launched, bounced in the dock, and had a UI pop up stating that it was installing dot net. I usually run installers and then go off and do other things. No one needs to watch paint dry.

Hmm…ok. So what happened? One of my biggest frustrations in general with products is a lack of log files or pointing people where to go.

Fortunately I have some experience to fall back on. I loaded up Suspicious Package and dropped in the Active Trader installer. There in the postinstall script we see a line that looks interesting:

$HOME/Library/Application Support/Active Trader Pro/Bottles/ActiveTraderPro64/drive_c/atp_install_progress.txt

Let’s check that file and folder out to see whats up:

open "$HOME/Library/Application Support/Active Trader Pro/Bottles/ActiveTraderPro64/drive_c/"

And the contents of atp_instal_progress.txt

Progress: 0
An error occured downloading Microsoft .NET 4.6.2!

First note. They are using Wine, or more specifically the commercial version CrossOver. So the app is native Windows, and then to make it available to macOS users they are bundling it up and serving it with CrossOver. Lots of thoughts on this but that is for another time. It is kind of fun to poke around at all the registry patches and other things they bundle. They even bundle netscape.exe! But I digress.

Ok! So we have something to search for. Let’s see if we can find where that error gets thrown. I’ll use ripgrep.

cd "/Applications/Active Trader Pro.app"
rg "occured downloading Microsoft .NET 4.6.2" . ./Contents/SharedSupport/activetrader/Active Trader Pro/run_atp
205:       echo "An error occured downloading Microsoft .NET 4.6.2!" >> "$OURBOTTLE"/drive_c/atp_install_progress.txt.new

Alright. So line 205 in ./Contents/SharedSupport/activetrader/Active Trader Pro/run_atp. Let’s take a look

#Install .NET 4.6.2
"$BIN_PATH/wine" --bottle=ActiveTraderPro64 --no-update --no-winewrapper --wl-app regedit "C:\\win7.reg"echo "Progress: 40" > "$OURBOTTLE"/drive_c/atp_install_progress.txt.new

echo "Downloading Microsoft .NET 4.6.2..." >> "$OURBOTTLE"/drive_c/atp_install_progress.txt.new

mv "$OURBOTTLE"/drive_c/atp_install_progress.txt.new "$OURBOTTLE"/drive_c/atp_install_progress.txtrm -f "$OURBOTTLE/drive_c/dotnet462.exe"

CX_LOG="$CX_LOG" CX_DEBUGMSG="$CX_DEBUGMSG$DBG_CONN" caffeinate -i "$BIN_PATH/wine" --bottle=ActiveTraderPro64 --no-update --no-winewrapper --wl-app cxwget --noui 'https://go.microsoft.com/fwlink/?linkid=2099468' "C:\\dotnet462.exe"

if [ $? != 0 ]; then
 echo "Progress: 0" > "$OURBOTTLE"/drive_c/atp_install_progress.txt.new
 echo "An error occured downloading Microsoft .NET 4.6.2!" >> "$OURBOTTLE"/drive_c/atp_install_progress.txt.new
 mv "$OURBOTTLE"/drive_c/atp_install_progress.txt.new "$OURBOTTLE"/drive_c/atp_install_progress.txt
 exit 1
fi

Hmm…seems pretty innocuous. Let’s verify that go.microsoft.com link downloads in my browser. Yep. And it’s the same link listed by MS here. Note that when it downloads it actually redirects and downloads this link:

https://download.visualstudio.microsoft.com/download/pr/8e396c75-4d0d-41d3-aea8-848babc2736a/80b431456d8866ebe053eb8b81a168b3/ndp462-kb3151800-x86-x64-allos-enu.exe

Looks like the command is referencing --wl-app cxwget which I am going to guess is just wget. Let’s find out:

find /Applications/Active\ Trader\ Pro.app -name "*cxwget*" -exec ls -al {} \;

:/$ find /Applications/Active\ Trader\ Pro.app -name "*cxwget*" -exec ls -al {} \;

-rwxr-xr-x  1 508  wheel  229216 Mar 27 07:57 /Applications/Active Trader Pro.app/Contents/SharedSupport/activetrader/lib64/wine/cxwget.exe

lrwxr-xr-x  1 508  wheel  25 May 15 12:54 /Applications/Active Trader Pro.app/Contents/SharedSupport/activetrader/lib32on64/wine/cxwget.exe -> ../../lib/wine/cxwget.exe

-rw-r--r--  1 508  wheel  206450 Mar 24 11:12 /Applications/Active Trader Pro.app/Contents/SharedSupport/activetrader/support/ActiveTraderPro64/drive_c/windows/syswow64/cxwget.exe

-rw-r--r--  1 508  wheel  220023 Mar 24 11:12 /Applications/Active Trader Pro.app/Contents/SharedSupport/activetrader/support/ActiveTraderPro64/drive_c/windows/system32/cxwget.exe

-rwxr-xr-x  1 508  wheel  215648 Mar 27 07:50 /Applications/Active Trader Pro.app/Contents/SharedSupport/activetrader/lib/wine/cxwget.exe

Let’s take a look:

strings /Applications/Active Trader Pro.app/Contents/SharedSupport/activetrader/lib64/wine/cxwget.exe | rg -i wget

Usage: cxwget [--resolve-redir | --noui] url

or: cxwget url file

Bummer. Doesn’t look like it supports versions or anything else. Google is also coming up short for cxwget.exe but cx is CrossOver and wget is well…wget.

Okay. Well hmm. What’s that CX_LOG var? Scrolling up to the top I see:

CX_LOG="$HOME"/Documents/ATP-$(date -j '+%Y-%m-%d-%H-%M-%S').log

Ah! Nice. A new log. Let’s go check that out:

cd $HOME/Documents && ls -al ATP-*;
bunzip2 ATP-2023-05-15-15-17-51.log.bz2  && st ATP-2023-05-15-15-17-51.log

Wow! It’s 150,000+ lines. Great. Let’s skip to the end and work backwards

1090038.178:0044:0045:trace:wininet:netconn_verify_cert verifying L"download.visualstudio.microsoft.com"

1090038.181:0044:0045:warn:wininet:netconn_verify_cert CERT_TRUST_IS_NOT_TIME_VALID

1090038.182:0044:0045:warn:wininet:netconn_verify_cert failed 12037

1090038.182:0044:0045:warn:wininet:netcon_secure_connect_setup cert verify failed: 12037

1090038.182:0044:0045:warn:wininet:netcon_secure_connect_setup Failed to establish SSL connection: 00000000 (12037)

1090038.182:0044:0045:warn:wininet:HTTP_HttpSendRequestW failed to upgrade to secure connection

Aha! So it looks like the https url redirect from go.microsoft.com is working correctly

1090037.752:0044:0045:trace:wininet:netconn_verify_cert verifying L"go.microsoft.com"
1090038.005:0044:0045:trace:wininet:netcon_secure_connect_setup established SSL connection

but for some reason it is unable to establish the ssl connection to the redirected download.visualstudio.microsoft.com.

When running the installer I had finder opened and noticed that the $HOME/Library/Application Support/Active Trader Pro/Bottles/ActiveTraderPro64/drive_c/ folder also contained a dotnet400.exe and line 166 in the run_atp script downloads it from fidelity:

CX_LOG="$CX_LOG" CX_DEBUGMSG="$CX_DEBUGMSG$DBG_CONN" caffeinate -i "$BIN_PATH/wine" --bottle=ActiveTraderPro64 --no-update --no-winewrapper --wl-app cxwget --noui 'https://www.fidelity.com/webcontent/ATPDotNet/dotNetFx40_Full_x86_x64.exe' "C:\\dotnet400.exe"

And if I scroll up that massive log file I can see it successfully negotiating and downloading

1090033.822:0039:003a:trace:wininet:netconn_verify_cert verifying L"www.fidelity.com"
1090034.074:0039:003a:trace:wininet:netcon_secure_connect_setup established SSL connection

Alright so it’s gotta be an ssl thing with cxwget. Right? But just to be sure let’s try my local wget to make sure it works.

:~/Desktop$ wget https://go.microsoft.com/fwlink/?linkid=2099468

--2023-05-15 22:07:32--  https://go.microsoft.com/fwlink/?linkid=2099468
Resolving go.microsoft.com (go.microsoft.com)... 96.7.78.134
Connecting to go.microsoft.com (go.microsoft.com)|96.7.78.134|:443... connected.

HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://download.visualstudio.microsoft.com/download/pr/8e396c75-4d0d-41d3-aea8-848babc2736a/80b431456d8866ebe053eb8b81a168b3/ndp462-kb3151800-x86-x64-allos-enu.exe [following]

--2023-05-15 22:07:32--  https://download.visualstudio.microsoft.com/download/pr/8e396c75-4d0d-41d3-aea8-848babc2736a/80b431456d8866ebe053eb8b81a168b3/ndp462-kb3151800-x86-x64-allos-enu.exe

Resolving download.visualstudio.microsoft.com (download.visualstudio.microsoft.com)... 93.184.215.201

Connecting to download.visualstudio.microsoft.com (download.visualstudio.microsoft.com)|93.184.215.201|:443... connected.

ERROR: cannot verify download.visualstudio.microsoft.com's certificate, issued by ‘CN=DigiCert SHA2 Secure Server CA,O=DigiCert Inc,C=US’:
  Issued certificate has expired.
To connect to download.visualstudio.microsoft.com insecurely, use `--no-check-certificate'.

Fail! Lets take a look at the cert:

date && openssl s_client --connect download.visualstudio.microsoft.com:443 2>/dev/null | openssl x509 -noout -dates
Mon May 15 22:15:09 PDT 2023
notBefore=Jul 11 00:00:00 2022 GMT
notAfter=Jul 11 23:59:59 2023 GMT

Looks fine. Let’s take a deeper look: openssl s_client --connect download.visualstudio.microsoft.com:443

There it is:

depth=1 C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA
verify error:num=10:certificate has expired
notAfter=Mar  8 12:00:00 2023 GMT
verify return:1

On March 8 DigiCert made some changes to their root and ICA certs. you can read more here.

what about curl? Yeah that works.

curl -L -o dotnet462.exe https://go.microsoft.com/fwlink/?linkid=2099468

Alright so what’s up with wget. Let’s checkout the filesystem while we run the wget command

sudo fs_usage -e iTerm2 -w | grep wget | grep -i ssl
stat64            /usr/local/etc/openssl@1.1/certs>>>>>>>>>>>>>>>>>>>>>>>>                         0.000021   wget

yep I’ve got openssl installed via homebrew. Deja vu creeping back in. The cert is installed a directory above in /usr/local/etc/ca-certificates/cert.pem and is actually part of another formula ca-certificates that is hosted by curl!

The file is from January. Alright let’s re-install ca-certificates

brew reinstall ca-certificates

Nice. wget working locally again. What about Fidelity? Nope. Easy searches did not reveal where they are stored within the emulated wine filesystem so I’m going to punt that to the Fidelity devs to figure out.

I just want to get trading not go down wine/custom wget/ssl rabbit holes so I will hook that curl command into the run_atp script. I suppose I could modify the script to download the file over http but that is not great for a lot of reasons.

comment out line 202 and then insert a new line 202 above it in /Applications/Active\ Trader\ Pro.app/Contents/SharedSupport/activetrader/Active\ Trader\ Pro/run_atp

curl -L -o "$OURBOTTLE/drive_c/dotnet462.exe" https://go.microsoft.com/fwlink/?linkid=2099468

I’m Finally Trading!

There are other ways to diagnose these sorts of things using tools like fs_usage, dtrace, dtruss, strace, ptrace, opensnoop, iosnoop, ps, lsof or a nice GUI like Instruments or ProcMon that are especially helpful when no logs exist. I have used them all over the years but this was my path this time.

Hopefully this was interesting and or helpful to someone else who is trying to troubleshoot a program on their computer, or who just wants to trade on Fidelity’s platform on macOS.

One more Thing

Dear Fidelity, I LOVE that you have detailed logs that let me figure out what is going on. It would be nice if they were not crowding my ~/Documents folder and instead were in a nice tidy place like within a folder in ~/Library/Logs/FidelityActiveTrader or something of the sort.

Extra Credit…

Down the rabbit hole:

Remember when I was looking into cxwget.exe above and ran strings? There was an interesting path in there: /Volumes/Home/piotr/crossover_atp/winelib/cxwget/win64

/Volumes could be anything but is generally associated with macOS. And following Home is piotr. This is probably a dev associated with this project. Usually I would expect first initial last name (P Iotr) however the name Piotr is also a first name found around the world but more commonly in Russia/Poland where there are some world class software developers. Let’s google

"piotr" codeweavers

Hey! Maybe this is him: https://www.codeweavers.com/about/people/piotr/ and I can figure out a way to get in touch.

I also tried “Piotr Fidelity” and found a Director of SaaS for FIS (Fidelity National Information Services, Inc). I typed https://fisglobal.com into chrome and got an ominous 400. NSA? what?

Tinfoil hat off I proceeded to www.fisglobal.com and learned that they have their hands in A LOT of money. So now I get to go learn something new and read another 10Q. Do you know anything about them? Do they share history with Fidelity? Such are rabbit holes.

Drop some comments below or send me a note.

Mindful Ruminations

Discussion about this post