{"id":1331,"date":"2025-05-02T00:00:00","date_gmt":"2025-05-01T22:00:00","guid":{"rendered":"https:\/\/demo.sidekick.be\/itenium\/ml-net-lightgbm-docker\/"},"modified":"2025-11-28T13:31:41","modified_gmt":"2025-11-28T12:31:41","slug":"ml-net-lightgbm-docker","status":"publish","type":"post","link":"https:\/\/demo.sidekick.be\/itenium\/blog\/docker\/ml-net-lightgbm-docker\/","title":{"rendered":"Training with ML.NET and LightGBM on Synology with Docker"},"content":{"rendered":"

Docker Training<\/h1>\n

As, once deployed, training failed with:<\/p>\n

System.DllNotFoundException<\/strong>: Unable to load shared library \u2018lib_lightgbm\u2019 or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you\u2019re using glibc, consider setting the LD_DEBUG environment variable:
libgomp.so.1: cannot open shared object file: No such file or directory
\/app\/lib_lightgbm.so: cannot open shared object file: No such file or directory<\/p>\n

It\u2019s a conscious decision by the team to not include LightGBM by default, and that makes perfect since it is a 1gig dependency which is only used when using that trainer.<\/p>\n

Installing LightGBM<\/h1>\n

A simple Google search got me to the LightGBM installation instructions:<\/p>\n

git clone --recursive https:\/\/github.com\/microsoft\/LightGBM.git\ncd LightGBM && mkdir build && cd build\ncmake .. && make -j && make install\n<\/code><\/pre>\n
CMake Detour<\/h2>\n
The apt-get install cmake -y<\/code> installed version 3.25.1 which was just not good enough for LightGBM!<\/p>\n
CMake 3.28 or higher is required. You are running version 3.25.1<\/p>\n
And now it was really starting to show that I\u2019m very far from a Linux installation\/compilation hero. After trying to build CMake for a few hours, I ended up installing pip<\/code> to install CMake with.<\/p>\n
apt-get install python3-pip -y\npip install cmake --upgrade --break-system-packages\n\n# Check version:\ncmake --version\n<\/code><\/pre>\n
The --break-system-packages<\/code> obviously isn\u2019t a very good sign but since we will be doing this in the build step of our Dockerfile, it\u2019s fine \ud83d\ude05<\/p>\n
Final Dockerfile<\/h1>\n
FROM mcr.microsoft.com\/dotnet\/sdk:8.0 AS build\n\nRUN apt-get update\nRUN apt-get install git clang-3.9 libunwind8 curl libomp-dev -y\n\nRUN apt-get install python3-pip -y\nRUN pip install cmake --upgrade --break-system-packages\n\nRUN git clone --recursive https:\/\/github.com\/microsoft\/LightGBM.git\nRUN cd LightGBM && mkdir build && cd build && cmake .. && make -j && make install\n\n# Usual steps for building the .NET app\n# dotnet restore\/publish etc\n\nFROM mcr.microsoft.com\/dotnet\/aspnet:8.0 as runtime\n\nCOPY --from=build \/app \/app\nCOPY --from=build \/usr\/local\/lib\/lib_lightgbm.so \/usr\/local\/lib\/lib_lightgbm.so\nCOPY --from=build \/usr\/lib\/x86_64-linux-gnu\/libgomp.so.1 \/usr\/lib\/x86_64-linux-gnu\/libgomp.so.1\n\nENTRYPOINT [ \"dotnet\", \"\/app\/WebApi.dll\" ]\n<\/code><\/pre>\n
I figured out which files needed to be copied with the following command:<\/p>\n
# Enter running container\n# (with lightgbm installed)\ndocker exec -it lightgbm bash\n\n# Find location\nfind \/ -name 'lib_lightgbm.so' 2>\/dev\/null\n\n# List dynamic dependencies\nldd \/usr\/local\/lib\/lib_lightgbm.so\n<\/code><\/pre>\n
And there we have it, a WebApi that creates the models!<\/p>\n
Extras<\/h1>\n
Stuff that came into being during the making of this post<\/b><\/p>\n