OBS not working with RTX 5090 (NV_ENC_ERR_INVALID_DEVICE)

T

Takibo

New Member
Hey!
I'm not entirely sure if this is going to be an isolated case or a bigger issue across other RTX 5090 Cards.
I'll gladly help find a solution for this & hopefully will help find a fix for other 50 Cards users as well.

I have spent over a week working around with OBS in an attempt to fix this, i'll list here below everything that was attempted as i tried to fix this issue.

Do note that, this issue does NOT occur with ANY other recording software, so the concerns for it being a hardware issue is highly unlikely, i am currently getting by with another recording software, as i do YouTube full time and couldn't afford to not find a solution ASAP.



First, the issue:

1. OBS crashes my recording midway through with the error: NV_ENC_ERR_INVALID_DEVICE
2. Crashes are not immediate, and may often take hours to appear (happen at random) ranging from as quick as 15 minutes into a recording to taking 8 hours to crash.


Second, useful information & all i've tested, etc:

1. This issue does NOT persist anywhere else (tested with Shadowplay, FFmpeg NVENC Rendering, Meld Studio)
2. This issue happens regardless of settings & 'Video Encoder' (anything NVENC err's out)
3. Stress testing does not accelerate nor does it deaccelerate the rate of these errors (tested increasing thermals, power draw, quality settings, no impact)
4. similar to #3, it does not matter whether im idle, or running benchmarks, the crashes happen all the same.
5. I have updated drivers, reinstalled utilizing DDU (multiple times), tested "studio drivers", also ran older drivers, nothing changed
6. I have attempted with rolling back OBS (and nvidia) no impact.
7. HAGS has been tested both off and on
8. High Priority has no impact
9. Running as admin has no impact
10. Power management has been tested
11. Have reseated the RTX 5090, also have changed the 16-pin power cable for another
12. Have tested limiting power draw, no impact
13. Matter of fact; thermals are always fine, power draw has standard fluctuations & CPU/SSD/RAM Behavior is always standard AFAIK.
14. Each and every ERROR gives me an "Event Viewer" Error: (Source: nvlddmkm / Event ID: 153) log attached
15. Does not seem to have VRAM or RAM leaks
16. There are likely more i've experimented with (its been over a week of straight experimenting) but this is about all i remember

I have attached here one of the various crash logs, from obs, and one from event viewers, there are a few others like it, but they always have the same readings so i wont fill you with many logs


For the time being i'll remain recording with another software, am looking forward to returning to OBS though..

Thanks!
 

Attachments

  • OBS LOG 2025-03-04 17-05-58.txt
    12.9 KB · Views: 85
  • event viewer crash.txt
    649 bytes · Views: 51
R

rockbottom

Active Member
You're not the only person having trouble with a 5090. Drivers don't seem to be all that great yet & are usually the cause of the encoder lockup. Have you tried running all your monitors @ 60HZ? Also, Nvidia released another driver on Wednesday (572.70), might be worth a shot.

Some other things, probably not contributing to the lock up but.
Disable Game DVR
Run all monitors @ 60HZ for testing
Set Color Range to Partial
Fix your bugged Scene
Use Preset P5
Set Keyframe Interval to 2 Seconds
Write to an mkv container while testing
 
T

Takibo

New Member
rockbottom said:
Sounds familiar?

Click to expand...
His issue seems to be different from mine, my load does not drop, matter of fact nothing seems out of place when the error occurs, games dont freeze, dont misbehave, dont drop in fps, it simply gives me the error on OBs and thats it, as if obs just lost communication or something witht the nvenc encoder
rockbottom said:
You're not the only person having trouble with a 5090. Drivers don't seem to be all that great yet & are usually the cause of the encoder lockup. Have you tried running all your monitors @ 60HZ? Also, Nvidia released another driver on Wednesday (572.70), might be worth a shot.

Some other things, probably not contributing to the lock up but.
Disable Game DVR
Run all monitors @ 60HZ for testing
Set Color Range to Partial
Fix your bugged Scene
Use Preset P5
Set Keyframe Interval to 2 Seconds
Write to an mkv container while testing
Click to expand...
Ill test the hz just for pure testing sakes and will return here later to share results! Thats about the only thing there i have not tested (also the color range, ill test that)

As for the drivers from yesterday, P5, game dvr, mkv, and keyframe interval all have been tested and none worked

Will drop results here with the 60hz and color range as soon as i can, im working on a few videos at the moment so ill probably run the experiment tomorrow!
 
R

rockbottom

Active Member
Many different issues over at that forum but hi refresh rates seem to be playing a part in some of the situations. Anyway, I'm still running the same version of OBS as you but I'm also running older hardware & 566.xx driver with no AV1 or issues. Looking at the release notes for the latest version of OBS, you may want to update since you're trying to encode AV1.
 
shata

shata

New Member
There is still no Fix for this issue on the latest Nvidia drivers, 576.02 with 5090. OBS needs to have a look at this. I have already tried reporting this Nvidia driver as a bug multiple times.
 
L

Limealicious

New Member
To add a data point, I started experiencing this issue after switching from a 4090 to a 5090. I did not change the driver but suddenly started getting the error 2-3 times a stream. The games I was playing didn't seem to have an issue and continued uninterrupted. Reinstalling drivers with DDU did not resolve the problem. Switching back to my 4090 immediately fixed the issue, and I'm still running the same driver as before. I'd suspect a bad card, but everything else was flawless (including heavy stress tests, low utilization situations, etc etc) and a number of my other friends in the creator space have had the issue specifically with 5090s. One of them has used every driver on his 5090FE since launch with no resolution, and the other has been having the problem across multiple drivers since MH Wilds launch roughly.

I've had to box my new toy until NVIDIA or OBS figures this out as I can't be nuking my viewer count multiple times a stream.
 
C

criken2

New Member
Same boat with my 5090 as everyone above. Tried multiple different driver versions and different encoders on OBS and they all run into the same NVENC GPU crash issue regardless. My backup card is a 3090 so I'm either going to have to go out and buy a 4090 till this gets fixed or put together a dual PC setup overnight
 

Attachments

  • obs64_g3PJID9Ha2.png
    obs64_g3PJID9Ha2.png
    400.9 KB · Views: 71
T

Takibo

New Member
Hey everybody, OP here.

Would just like to add that ive found NO fix.

Ive tested many more new things to no avail. It seems to be an nvidia driver issue.

Meld studios does not hard crash like OBS so ive been using that.. when the ‘crash’ happens the stream freezes for 10 ish seconds but at least doesnt hard crash like obs. I know its probably not obs’ fault but yeah
 
Jadenb115

Jadenb115

New Member
Takibo said:
Hey everybody, OP here.

Would just like to add that ive found NO fix.

Ive tested many more new things to no avail. It seems to be an nvidia driver issue.

Meld studios does not hard crash like OBS so ive been using that.. when the ‘crash’ happens the stream freezes for 10 ish seconds but at least doesnt hard crash like obs. I know its probably not obs’ fault but yeah
Click to expand...
Same issues Rtx 5090 and ryzen 9 9950X3D. I have submitted the issue on NVIDIA's driver support forums. I doubt they will even see it. I wish they would fix their crap.
 
Archaic

Archaic

New Member
Jadenb115 said:
Same issues Rtx 5090 and ryzen 9 9950X3D. I have submitted the issue on NVIDIA's driver support forums. I doubt they will even see it. I wish they would fix their crap.
Click to expand...
If the issue is only with OBS when Meld has been reported as working just fine with those having issues, wouldnt that mean the issue is on OBS's side?
 
M

masoh

New Member
5070ti/5700x3d

This has been happening to a lot of people with Windows 11 and a 40/50s series card. Mainly AMD CPU's as well.

Everyone look in the Windows Event Viewer while the crash happens. You should be seeing event nvlddmkm error 153. Lots of different forums talking about this mysterious problem. Definitely an NVIDIA/Windows issue but it can mean so many different things.

I have struggled with this since I've had my 5070ti and had to get these new drivers. Tested many different solutions. Can only reproduce the event by doing OCCT VRAM test. Have tried the OCCT VRAM test on my 3070 and it also popped the code. Makes me think its not the GPU. I want to try to stream with it to see if it(3070) crashes, but OBS wont work when I plug in my 3070 again. Ill try to get it to work and update.

After paying almost 1k for a single piece of hardware I have never been so annoyed. OBS is force closed every time it happens and disconnect protection does nothing because it stops the stream. You just can't be having that when you have a good bit of viewers or you get a nice raid.

I've lost COUNTLESS hours and have read every forum post out there trying to figure this out. I even started swapping parts from my old PC to see if I could narrow it down and in the process I have somehow messed up my old rig. The motherboard will not power up anymore, it just blinks on and quits. So in total I bought a new PC(700$) to put my 5070ti(900$) in. LOST my entire old PC(Mainly my fault, but wouldn't be doing this in the first place if it wasn't for this) in the process AND now have a crashing PC lol.

Mess of a post and sorry for the rant, I am so fed up and disappointed. I am either going to hard reset everything in my system and get Windows 10 and try, or thinking about just ordering a brand new CPU+MOBO+SSD and downgrading to Windows 10 for science. I really feel like there are either some corrupted windows files(Which it says there not but who knows) or just bad NVIDIA drivers(Have DDU'd and reinstalled the last 4 updates).
 
P

prgmitchell

Forum Moderator
masoh said:
5070ti/5700x3d

This has been happening to a lot of people with Windows 11 and a 40/50s series card. Mainly AMD CPU's as well.

Everyone look in the Windows Event Viewer while the crash happens. You should be seeing event nvlddmkm error 153. Lots of different forums talking about this mysterious problem. Definitely an NVIDIA/Windows issue but it can mean so many different things.

I have struggled with this since I've had my 5070ti and had to get these new drivers. Tested many different solutions. Can only reproduce the event by doing OCCT VRAM test. Have tried the OCCT VRAM test on my 3070 and it also popped the code. Makes me think its not the GPU. I want to try to stream with it to see if it(3070) crashes, but OBS wont work when I plug in my 3070 again. Ill try to get it to work and update.

After paying almost 1k for a single piece of hardware I have never been so annoyed. OBS is force closed every time it happens and disconnect protection does nothing because it stops the stream. You just can't be having that when you have a good bit of viewers or you get a nice raid.

I've lost COUNTLESS hours and have read every forum post out there trying to figure this out. I even started swapping parts from my old PC to see if I could narrow it down and in the process I have somehow messed up my old rig. The motherboard will not power up anymore, it just blinks on and quits. So in total I bought a new PC(700$) to put my 5070ti(900$) in. LOST my entire old PC(Mainly my fault, but wouldn't be doing this in the first place if it wasn't for this) in the process AND now have a crashing PC lol.

Mess of a post and sorry for the rant, I am so fed up and disappointed. I am either going to hard reset everything in my system and get Windows 10 and try, or thinking about just ordering a brand new CPU+MOBO+SSD and downgrading to Windows 10 for science. I really feel like there are either some corrupted windows files(Which it says there not but who knows) or just bad NVIDIA drivers(Have DDU'd and reinstalled the last 4 updates).
Click to expand...

NVIDIA is aware of the issue and it seems to be on their end, nothing you can do besides waiting for a fix from them via a driver release.
 
D

DG_AU

New Member
Any updates on this? I can update to the latest 576.40 but it doesn't state anything of nvlddmkm, even in the known issues, are we sure nvidia is aware of this? I also get the nvlddmkm 153 error, I used to get black screens and gpu fans go 100%, but that seemed to go away, but now I'm getting this exact same issue as you with OBS now. (Astral 5090)

I just started getting this error 2 days ago, think it might've been a windows update that stuffed something up not sure I did anything else, still on driver 576.28. Any ideas? Just hoping it isn't a hardware issue, I've done A LOT of troubleshooting, software and hardware wise
 
P

prgmitchell

Forum Moderator
DG_AU said:
Any updates on this? I can update to the latest 576.40 but it doesn't state anything of nvlddmkm, even in the known issues, are we sure nvidia is aware of this? I also get the nvlddmkm 153 error, I used to get black screens and gpu fans go 100%, but that seemed to go away, but now I'm getting this exact same issue as you with OBS now. (Astral 5090)

I just started getting this error 2 days ago, think it might've been a windows update that stuffed something up not sure I did anything else, still on driver 576.28. Any ideas? Just hoping it isn't a hardware issue, I've done A LOT of troubleshooting, software and hardware wise
Click to expand...

Yes we are sure NVIDIA is aware of this.
 
M

masoh

New Member
I will probably jinx it here, but since the most recent driver update I have gotten no crashes in 5 days(30 hours streamed). Have also gotten a 15 day streak in the past randomly with lots of hours streamed but it came back eventually. Time will tell. Will update when I crash. Anyone else have success with these new drivers?
 
D

DG_AU

New Member
masoh said:
I will probably jinx it here, but since the most recent driver update I have gotten no crashes in 5 days(30 hours streamed). Have also gotten a 15 day streak in the past randomly with lots of hours streamed but it came back eventually. Time will tell. Will update when I crash. Anyone else have success with these new drivers?
Click to expand...
Are you on 576.52 or 576.40? I am currently on 576.28, should I update? + Did you use the nvidia app or use DDU and install driver manually? Studio or game ready driver?
 
Last edited:
You must log in or register to reply here.
Top