new kernel 6.5.0-14(22.04) can not compile NVIDIA display card driver

[ubfan1]

The Nvidia 535.146.02 driver is being "phased", so is likely not to be upgraded with the 6.5 kernel. You can check with apt policy nvidia-driver-535.
You can force the upgrade with sudo apt -o APT::Get::Always-Include-Phased-Updates=true upgrade

[sakalaboator]

You seem to be set just on using the NVidia Binary drivers from Nvidia that are not working for you. You do not seem to want to try to compile it with gcc-12 to see if that works or not.

From what I read, it works if compiled with gcc-12...

I have an idea... Post on the NVidia Forum and see what they say... It's their driver = Third party non-free.

Thanks, I have post the problem on NV forum, some user also has the same problem. Now I am waiting for the NV driver update. Actually, gcc version is very important for other apps, so I will not try gcc12.

[sakalaboator]

The Nvidia 535.146.02 driver is being "phased", so is likely not to be upgraded with the 6.5 kernel. You can check with apt policy nvidia-driver-535.
You can force the upgrade with sudo apt -o APT::Get::Always-Include-Phased-Updates=true upgrade

Thanks, I decide to wait for new NV driver update, maybe 536, or 540 will come out in these days.

[raphaelnnoe]

I'm struggling with this same problem, after the kernel update the nvidia drive stopped working, and in trying to solve it I ended up breaking the system, I formatted it but the problem persisted.
This big **** from canonical and nvidia **** me all over.

[MAFoElffen]

Didn't some GPU's end support at 525, 535, & 540?

IDK.

Doing NVidia support is work. I admit. Usually someone tells me the GPU... Then you have to look at these
https://forums.developer.nvidia.com/...se-case/249977
https://www.nvidia.com/Download/index.aspx?lang=en-us
https://www.nvidia.com/en-us/drivers/unix/

To play sleuth to figure out which driver a GPU uses and where the support ended.

I wish there was "a chart" that just was by GPU, and listed the last Driver Version was for that GPU! But I guess that makes too much sense.

[sakalaboator]

Thanks Peshkopy in nvidia forum, Thanks for all. This error is resolved by gcc version update. Now it works well.

1. gcc -v
   Using built-in specs.
   COLLECT_GCC=gcc
   COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/12/lto-wrapper
   OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
   OFFLOAD_TARGET_DEFAULT=1
   Target: x86_64-linux-gnu
   Configured with: …/src/configure -v --with-pkgversion=‘Ubuntu 12.3.0-1ubuntu1~22.04’ --with-bugurl=file:///usr/share/doc/gcc-12/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-12 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --enable-cet --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-12-ALHxjy/gcc-12-12.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-ALHxjy/gcc-12-12.3.0/debian/tmp-gcn/usr --enable-offload-defaulted --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
   Thread model: posix
   Supported LTO compression algorithms: zlib zstd
   gcc version 12.3.0 (Ubuntu 12.3.0-1ubuntu1~22.04)
2. nvidia-smi
   Mon Jan 15 09:19:11 2024
   ±--------------------------------------------------------------------------------------+
   | NVIDIA-SMI 535.146.02 Driver Version: 535.146.02 CUDA Version: 12.2 |
3. Some old drivers eg. 390, can not supported by GCC12.

[MAFoElffen]

Thanks Peshkopy in nvidia forum, Thanks for all. This error is resolved by gcc version update. Now it works well.

I said this in Post #6... And explained "the why". That Kernel 6.5.0-14 was compiled with ggc-12, so that the drivers need to be matched to that... When I said that, a few here said they didn't want to go that way. LOL. I understand that. Mantic & Noble do not have a problem, but those are already using newer gcc-12 by default. I didn't realize then, that older legacy drivers (304, 390, 410) would still not build with Ubuntu 22.04 & gcc-12.

I now have one old laptop with nvidia-driver-390 with 6.5 removed and pinned to 6.2 for that machine. But my other machines are running fine with nvidia-driver-535... I can post those notes for those interested, and with old legacy drivers... I already have that posted elsewhere here on the forum.

My install of gcc-12 is a bit more complete than was posted back in that NVidia forums thread... If you want to see how I did that, I'll post it from my admin journal notes.

[sakalaboator]

I said this in Post #6... And explained "the why". That Kernel 6.5.0-14 was compiled with ggc-12, so that the drivers need to be matched to that... When I said that, a few here said they didn't want to go that way. LOL. I understand that. Mantic & Noble do not have a problem, but those are already using newer gcc-12 by default. I didn't realize then, that older legacy drivers (304, 390, 410) would still not build with Ubuntu 22.04 & gcc-12.

I now have one old laptop with nvidia-driver-390 with 6.5 removed and pinned to 6.2 for that machine. But my other machines are running fine with nvidia-driver-535... I can post those notes for those interested, and with old legacy drivers... I already have that posted elsewhere here on the forum.

My install of gcc-12 is a bit more complete than was posted back in that NVidia forums thread... If you want to see how I did that, I'll post it from my admin journal notes.

You are right! Thanks!

[MAFoElffen]

If now solved: Please visit the "Thread Tools" link in the upper right of the page & mark this thread as "Solved"... So that other may find your solution to help them with resolving this problem.

I have a felling that within the next few years, with how this has been going with kernel series changes, and now the announcement that LTS versions are going to be getting longer support... I'm just going to post both my fixes in this post, so that other may be able to find a solution to their similar problem. Thank you for the opportunity to look into this and helping. That way, we both can share how to solve this for others. I'll get the Moderators to add the link to this post to Post #2 of my Graphics Resolution Sticky.

To check which Compiler the system is using and compare it to what the current kernel was compiled with:

grep -e 'linux-gnu-gcc-' /proc/version
uname -r
gcc --version | grep -e 'gcc'

To install and change gcc compilers:

NVidia drivers sometime have a problem compiling the drivers if there is too much of a difference between the installed compiler and the version of compiler the running kernel was compiled with... Solution- install that compiler. This works right now for recently supported drivers (with kernel 6.5.0), but not currently for legacy drivers. For that work-around, skip this, and see below to run the working kernel...

Install the matching compiler:

sudo apt update
sudo apt install gcc-12 g++-12

Look to see which compiler pieces are installed

apt list g{cc,++}-*{0,1,2,3,4,5,6,7,8,9} --installed

For example, mine:

mafoelffen@msi-ubuntu:~$ apt list g{cc,++}-*{0,1,2,3,4,5,6,7,8,9} --installed
Listing... Done
g++-11/jammy-updates,jammy-security,now 11.4.0-1ubuntu1~22.04 amd64 [installed,automatic]
g++-12/jammy-updates,jammy-security,now 12.3.0-1ubuntu1~22.04 amd64 [installed]
gcc-11/jammy-updates,jammy-security,now 11.4.0-1ubuntu1~22.04 amd64 [installed,automatic]
gcc-12/jammy-updates,jammy-security,now 12.3.0-1ubuntu1~22.04 amd64 [installed,automatic]

mafoelffen@msi-ubuntu:~$ ls -larth `find /usr/bin/ -name gcc-*` | grep -v 'nm\|ar\|ra'
lrwxrwxrwx 1 root root 23 May 13  2023 /usr/bin/gcc-12 -> x86_64-linux-gnu-gcc-12
lrwxrwxrwx 1 root root 23 May 13  2023 /usr/bin/gcc-11 -> x86_64-linux-gnu-gcc-11

mafoelffen@msi-ubuntu:~$ ls -larth `find /usr/bin/ -name g++-*` | grep -v 'nm\|ar\|ra'
lrwxrwxrwx 1 root root 23 May 13  2023 /usr/bin/g++-12 -> x86_64-linux-gnu-g++-12
lrwxrwxrwx 1 root root 23 May 13  2023 /usr/bin/g++-11 -> x86_64-linux-gnu-g++-11

Add them to an update-alternatives group

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 11 --slave /usr/bin/g++ g++ /usr/bin/g++-11 --slave /usr/bin/gcov gcov /usr/bin/gcov-11
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12 --slave /usr/bin/g++ g++ /usr/bin/g++-12 --slave /usr/bin/gcov gcov /usr/bin/gcov-12

Temporarily pick gcc-12

mafoelffen@msi-ubuntu:~$ sudo update-alternatives --config gcc

There are 2 choices for the alternative gcc (providing /usr/bin/gcc).

  Selection    Path             Priority   Status
------------------------------------------------------------
* 0            /usr/bin/gcc-12   12        auto mode
  1            /usr/bin/gcc-11   11        manual mode
  2            /usr/bin/gcc-12   12        manual mode

Press <enter> to keep the current choice[*], or type selection number: 2

Reinstall my driver

sudo apt install --yes nvidia-driver-535 nvidia-dkms-535

Optionally, install Nvidia Cuda

sudo apt install --yes nvidia-cuda-toolkit

Reboot to test

After you are through, rest the compiler back to what it was...

mafoelffen@msi-ubuntu:~$ sudo update-alternatives --config gcc
There are 2 choices for the alternative gcc (providing /usr/bin/gcc).

  Selection    Path             Priority   Status
------------------------------------------------------------
  0            /usr/bin/gcc-12   12        auto mode
  1            /usr/bin/gcc-11   11        manual mode
* 2            /usr/bin/gcc-12   12        manual mode

Press <enter> to keep the current choice[*], or type selection number: 1
update-alternatives: using /usr/bin/gcc-11 to provide /usr/bin/gcc (gcc) in manual mode

If this work-around helped, then join this Bug Report as "Also Affects Me": https://ubuntuforums.org/showthread....4#post14175164

---

Updated: For NVidia legacy driver 390 for Jammy and Mantic, use this PPA (credit Yellow Pasque): https://launchpad.net/~dtl131/+archi...aexp/+packages
Tested. Works with 6.5.0 series kernels.

Old outdated work-around
Legacy Drivers: Use 6.2.0 series kernels...

sudo apt remove --purge --yes linux-headers-6.5.0-14-generic linux-image-6.5.0-14-generic linux-modules-6.5.0-14-generic linux-modules-extra-6.5.0-14-generic
sudo apt install --reinstall --yes linux-headers-6.2.0-39-generic linux-image-6.2.0-39-generic linux-modules-6.2.0-39-generic linux-modules-extra-6.2.0-39-generic
sudo apt-mark hold linux-headers-6.2.0-39-generic linux-image-6.2.0-39-generic linux-modules-6.2.0-39-generic linux-modules-extra-6.2.0-39-generic

When they come up with a fix to compile NVidia Legacy drivers with Linux kernel 6.5 series kernels, then you can re-add that kernel series by unpinning the kernels via

sudo apt-mark unhold linux-headers-6.2.0-39-generic linux-image-6.2.0-39-generic linux-modules-6.2.0-39-generic linux-modules-extra-6.2.0-39-generic
sudo apt update && sudo apt upgrade

[johndpope]

almost helpful - but hit a snag - need step 0 to be remove existing alternatives.

error - sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 11 --slave /usr/bin/g++ g++ /usr/bin/g++-11 --slave /usr/bin/gcov gcov /usr/bin/gcov-11sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12 --slave /usr/bin/g++ g++ /usr/bin/g++-12 --slave /usr/bin/gcov gcov /usr/bin/gcov-12 -(torch2) ➜ ~ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12 --slave /usr/bin/g++ g++ /usr/bin/g++-12 --slave /usr/bin/gcov gcov /usr/bin/gcov-12update-alternatives: error: alternative g++ can't be slave of gcc: it is a master alternative


sudo update-alternatives --remove gcc /usr/bin/gcc-11sudo update-alternatives --remove gcc /usr/bin/gcc-12sudo update-alternatives --remove g++ /usr/bin/g++-11sudo update-alternatives --remove g++ /usr/bin/g++-12

If now solved: Please visit the "Thread Tools" link in the upper right of the page & mark this thread as "Solved"... So that other may find your solution to help them with resolving this problem.

I have a felling that within the next few years, with how this has been going with kernel series changes, and now the announcement that LTS versions are going to be getting longer support... I'm just going to post both my fixes in this post, so that other may be able to find a solution to their similar problem. Thank you for the opportunity to look into this and helping. That way, we both can share how to solve this for others. I'll get the Moderators to add the link to this post to Post #2 of my Graphics Resolution Sticky.

To check which Compiler the system is using and compare it to what the current kernel was compiled with:

grep -e 'linux-gnu-gcc-' /proc/version
uname -r
gcc --version | grep -e 'gcc'

To install and change gcc compilers:

NVidia drivers sometime have a problem compiling the drivers if there is too much of a difference between the installed compiler and the version of compiler the running kernel was compiled with... Solution- install that compiler. This works right now for recently supported drivers (with kernel 6.5.0), but not currently for legacy drivers. For that work-around, skip this, and see below to run the working kernel...

Install the matching compiler:

sudo apt update
sudo apt install gcc-12 g++-12

Look to see which compiler pieces are installed

apt list g{cc,++}-*{0,1,2,3,4,5,6,7,8,9} --installed

For example, mine:

mafoelffen@msi-ubuntu:~$ apt list g{cc,++}-*{0,1,2,3,4,5,6,7,8,9} --installed
Listing... Done
g++-11/jammy-updates,jammy-security,now 11.4.0-1ubuntu1~22.04 amd64 [installed,automatic]
g++-12/jammy-updates,jammy-security,now 12.3.0-1ubuntu1~22.04 amd64 [installed]
gcc-11/jammy-updates,jammy-security,now 11.4.0-1ubuntu1~22.04 amd64 [installed,automatic]
gcc-12/jammy-updates,jammy-security,now 12.3.0-1ubuntu1~22.04 amd64 [installed,automatic]

mafoelffen@msi-ubuntu:~$ ls -larth `find /usr/bin/ -name gcc-*` | grep -v 'nm\|ar\|ra'
lrwxrwxrwx 1 root root 23 May 13  2023 /usr/bin/gcc-12 -> x86_64-linux-gnu-gcc-12
lrwxrwxrwx 1 root root 23 May 13  2023 /usr/bin/gcc-11 -> x86_64-linux-gnu-gcc-11

mafoelffen@msi-ubuntu:~$ ls -larth `find /usr/bin/ -name g++-*` | grep -v 'nm\|ar\|ra'
lrwxrwxrwx 1 root root 23 May 13  2023 /usr/bin/g++-12 -> x86_64-linux-gnu-g++-12
lrwxrwxrwx 1 root root 23 May 13  2023 /usr/bin/g++-11 -> x86_64-linux-gnu-g++-11

Add them to an update-alternatives group

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 11 --slave /usr/bin/g++ g++ /usr/bin/g++-11 --slave /usr/bin/gcov gcov /usr/bin/gcov-11
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-12 12 --slave /usr/bin/g++ g++ /usr/bin/g++-12 --slave /usr/bin/gcov gcov /usr/bin/gcov-12

Temporarily pick gcc-12

mafoelffen@msi-ubuntu:~$ sudo update-alternatives --config gcc

There are 2 choices for the alternative gcc (providing /usr/bin/gcc).

  Selection    Path             Priority   Status
------------------------------------------------------------
* 0            /usr/bin/gcc-12   12        auto mode
  1            /usr/bin/gcc-11   11        manual mode
  2            /usr/bin/gcc-12   12        manual mode

Press <enter> to keep the current choice[*], or type selection number: 2

Reinstall my driver

sudo apt install --yes nvidia-driver-535 nvidia-dkms-535

Optionally, install Nvidia Cuda

sudo apt install --yes nvidia-cuda-toolkit

Reboot to test

After you are through, rest the compiler back to what it was...

mafoelffen@msi-ubuntu:~$ sudo update-alternatives --config gcc
There are 2 choices for the alternative gcc (providing /usr/bin/gcc).

  Selection    Path             Priority   Status
------------------------------------------------------------
  0            /usr/bin/gcc-12   12        auto mode
  1            /usr/bin/gcc-11   11        manual mode
* 2            /usr/bin/gcc-12   12        manual mode

Press <enter> to keep the current choice[*], or type selection number: 1
update-alternatives: using /usr/bin/gcc-11 to provide /usr/bin/gcc (gcc) in manual mode

Legacy Drivers: Use 6.2.0 series kernels...

sudo apt remove --purge --yes linux-headers-6.5.0-14-generic linux-image-6.5.0-14-generic linux-modules-6.5.0-14-generic linux-modules-extra-6.5.0-14-generic
sudo apt install --reinstall --yes linux-headers-6.2.0-39-generic linux-image-6.2.0-39-generic linux-modules-6.2.0-39-generic linux-modules-extra-6.2.0-39-generic
sudo apt-mark hold linux-headers-6.2.0-39-generic linux-image-6.2.0-39-generic linux-modules-6.2.0-39-generic linux-modules-extra-6.2.0-39-generic

When they come up with a fix to compile NVidia Legacy drivers with Linux kernel 6.5 series kernels, then you can re-add that kernel series by unpinning the kernels via

sudo apt-mark unhold linux-headers-6.2.0-39-generic linux-image-6.2.0-39-generic linux-modules-6.2.0-39-generic linux-modules-extra-6.2.0-39-generic
sudo apt update && sudo apt upgrade