Stage 4, day 6:
I took a small break to take care of all the accumulated unrelated tasks to the trading project.
The excuse for the break was that I had some fund stuck in PAXG that I wanted to transfer back into BTC at a specific price to not realize a loss. Not sure if it is Thursday or yesterday but the wind has changed direction. BTC took a dive and gold did soar and that was what I needed to exit PAXG.
In the meantime, I did integrate a set of improvements into the code and I should be ready to do a second test run soon.
One of the task that I wanted to accomplish is to compile the code on a webserver that I have access to. That alone has been a fantastic journey (not 100% completed yet).
The challenge of that task was that my code is built on top of recent libraries while the server is running on software that is almost not supported anymore (CentOS 6 with kernel v2.6.32, glibc 2.12, gcc 4.4.7 and openssl 1.0.1e)
I needed to recompile openssl 1.1 (took the latest one)
compile a JSON lib
compile a websocket lib
On openssl, the installation script complain that the old compiler isn't supporting 128 bits variables, therefore, it will turn off some features. Oh well. What can I do.
Here comes the JSON lib turn.
This is where I did put my finger in the sw dependency hell gears.
Installation scripts refuse to run unless I upgrade autoconf. Start to download autoconf. Then this one complains about automake.
Automake want a libtool upgrade and I had to upgrade texinfo and help2man as well for some obscure reasons.
Ouf, I have been able to close back the pandora box for few moments.
websocket lib building has been painless.
Now, I reach the part where I try to build my own code. On the first compiled unit, the compiler stops and report errors.
It complains about something that I do in my code that have been valid for some time but not at the time that this compiler has been built. At first, I say to myself I'm going slightly change my code to satisfy this old compiler. Probably the fastest way to address this.
The second reported problem is more serious. Some glibc function that I call isn't in the old system glibc version. From prior experience, I know that it is very perilous to manually upgrade glibc on a old system. The reason being that every single program on the system is using that library. If you do a manipulation error with the upgrade, you essentially brick the box because nothing will work anymore.
A safer approach is simply to create a private version and just link the process that needs it.
https://stackoverflow.com/questions/8471...ingle-host
So, I download the latest version: 2.31
Installation script complains and tell me that I'll need the following component upgrades:
- binutils >= 2.25
- gmake >= 4.0
- bison >= 2.7
- python >= 3.4
- gcc >= 6.2
That is the real dependency hell that I'm talking about....
I took care of them one by one. The crunchiest one was the gcc compiler (9.3) which also has its own dependency requirement:
- gmp
- mpfr
- mpc
After hours of compiling and addressing dependency requirements, I finally end-up in succeeding compiling the latest glibc version (2.31) with the latest compiler (9.3) on a 10years old barely supported OS!
Moment of truth, I try the new glibc. At runtime, all it does is print out an error message:
FATAL: Kernel too old
Are you serious?? This after having spent a full night working on building all the required dependencies this is only after all this trouble that you tell me that I can't run this thing on my OS??
A small check by the installation script and letting me know about this upfront would have been appreciated...
Ok, after a small google search, I did figure out that the last glibc version that still support my kernel is 2.25 (2017) which is still a giant leap in terms of performance, functionality and bug fixes over the current 2.12 version on my system.
Ok, I start to compile 2.25 and its build is abruptly interrupted in the middle. After some research, I discover that a patch has been created for that problem:
https://sourceware.org/git/gitweb.cgi?p=...905638b797
This is unbelievable. How can you release software without even verifying that it, at least, compile fine out of the box... Anyway, I apply the patch and restart the build....
Finally, I got this upgrade:
My attempt to replace glibc only for my app did work. I did upgrade:
# /lib64/libc.so.6
GNU C Library stable release version 2.12, by Roland McGrath et al.
Copyright © 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.4.7 20120313 (Red Hat 4.4.7-23).
Compiled on a Linux 2.6.32 system on 2019-04-09.
Available extensions:
The C stubs add-on version 2.1.2.
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
RT using linux kernel aio
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>
to:
# /home/juicingf/dev/glibc-install/home/juicingf/lib/libc.so.6
GNU C Library (GNU libc) stable release version 2.25, by Roland McGrath et al.
Copyright © 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 9.3.0.
Available extensions:
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>;.
It works so well. That I think that I'm going to push my luck and create some rpms to make a global upgrade of glibc on the whole box. This could give some substantial kick to performance on the whole box. I'm going, of course, test my rpms on a crash test dummy VM. The last thing that I want is to brick the server... This is a real possibility when you mess with glibc... (I know because I did it in the past...)
Bottomline, This is very satisfying experience. That was a very hard challenge. I did share my project on the websocket lib mailing list when I had issues with it and openssl. The lib creator simply told me that what I was trying to do was impossible. Hopefully, all the nitty gritty details give some sense on the toughness of the task.
To say the least, it was ambitious. So happy to have made it run. Despite being a hard task, it has felt fun and easy to me... Probably some BASE induced state of mind did help me in succeeding...
I took a small break to take care of all the accumulated unrelated tasks to the trading project.
The excuse for the break was that I had some fund stuck in PAXG that I wanted to transfer back into BTC at a specific price to not realize a loss. Not sure if it is Thursday or yesterday but the wind has changed direction. BTC took a dive and gold did soar and that was what I needed to exit PAXG.
In the meantime, I did integrate a set of improvements into the code and I should be ready to do a second test run soon.
One of the task that I wanted to accomplish is to compile the code on a webserver that I have access to. That alone has been a fantastic journey (not 100% completed yet).
The challenge of that task was that my code is built on top of recent libraries while the server is running on software that is almost not supported anymore (CentOS 6 with kernel v2.6.32, glibc 2.12, gcc 4.4.7 and openssl 1.0.1e)
I needed to recompile openssl 1.1 (took the latest one)
compile a JSON lib
compile a websocket lib
On openssl, the installation script complain that the old compiler isn't supporting 128 bits variables, therefore, it will turn off some features. Oh well. What can I do.
Here comes the JSON lib turn.
This is where I did put my finger in the sw dependency hell gears.
Installation scripts refuse to run unless I upgrade autoconf. Start to download autoconf. Then this one complains about automake.
Automake want a libtool upgrade and I had to upgrade texinfo and help2man as well for some obscure reasons.
Ouf, I have been able to close back the pandora box for few moments.
websocket lib building has been painless.
Now, I reach the part where I try to build my own code. On the first compiled unit, the compiler stops and report errors.
It complains about something that I do in my code that have been valid for some time but not at the time that this compiler has been built. At first, I say to myself I'm going slightly change my code to satisfy this old compiler. Probably the fastest way to address this.
The second reported problem is more serious. Some glibc function that I call isn't in the old system glibc version. From prior experience, I know that it is very perilous to manually upgrade glibc on a old system. The reason being that every single program on the system is using that library. If you do a manipulation error with the upgrade, you essentially brick the box because nothing will work anymore.
A safer approach is simply to create a private version and just link the process that needs it.
https://stackoverflow.com/questions/8471...ingle-host
So, I download the latest version: 2.31
Installation script complains and tell me that I'll need the following component upgrades:
- binutils >= 2.25
- gmake >= 4.0
- bison >= 2.7
- python >= 3.4
- gcc >= 6.2
That is the real dependency hell that I'm talking about....
I took care of them one by one. The crunchiest one was the gcc compiler (9.3) which also has its own dependency requirement:
- gmp
- mpfr
- mpc
After hours of compiling and addressing dependency requirements, I finally end-up in succeeding compiling the latest glibc version (2.31) with the latest compiler (9.3) on a 10years old barely supported OS!
Moment of truth, I try the new glibc. At runtime, all it does is print out an error message:
FATAL: Kernel too old
Are you serious?? This after having spent a full night working on building all the required dependencies this is only after all this trouble that you tell me that I can't run this thing on my OS??
A small check by the installation script and letting me know about this upfront would have been appreciated...
Ok, after a small google search, I did figure out that the last glibc version that still support my kernel is 2.25 (2017) which is still a giant leap in terms of performance, functionality and bug fixes over the current 2.12 version on my system.
Ok, I start to compile 2.25 and its build is abruptly interrupted in the middle. After some research, I discover that a patch has been created for that problem:
https://sourceware.org/git/gitweb.cgi?p=...905638b797
This is unbelievable. How can you release software without even verifying that it, at least, compile fine out of the box... Anyway, I apply the patch and restart the build....
Finally, I got this upgrade:
My attempt to replace glibc only for my app did work. I did upgrade:
# /lib64/libc.so.6
GNU C Library stable release version 2.12, by Roland McGrath et al.
Copyright © 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.4.7 20120313 (Red Hat 4.4.7-23).
Compiled on a Linux 2.6.32 system on 2019-04-09.
Available extensions:
The C stubs add-on version 2.1.2.
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
RT using linux kernel aio
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>
to:
# /home/juicingf/dev/glibc-install/home/juicingf/lib/libc.so.6
GNU C Library (GNU libc) stable release version 2.25, by Roland McGrath et al.
Copyright © 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 9.3.0.
Available extensions:
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>;.
It works so well. That I think that I'm going to push my luck and create some rpms to make a global upgrade of glibc on the whole box. This could give some substantial kick to performance on the whole box. I'm going, of course, test my rpms on a crash test dummy VM. The last thing that I want is to brick the server... This is a real possibility when you mess with glibc... (I know because I did it in the past...)
Bottomline, This is very satisfying experience. That was a very hard challenge. I did share my project on the websocket lib mailing list when I had issues with it and openssl. The lib creator simply told me that what I was trying to do was impossible. Hopefully, all the nitty gritty details give some sense on the toughness of the task.
To say the least, it was ambitious. So happy to have made it run. Despite being a hard task, it has felt fun and easy to me... Probably some BASE induced state of mind did help me in succeeding...
