realvr
patreon
S is for Santa
🕑 Added 2024-12-21 22:37:27 +0000 UTCOf course! Ho ho ho ho... Have you been good?
I'm pretty sure that you have, so, as is his custom, Santa is going to bring you something that you had been asking for, but also something unexpected.
Some of you have guessed it already. S is for Santa, but it's also short for Stereoscopic. And for Space, which is where we'll be going with our next adventure []-D
Actually we've just been to space, with Star Wars Outlaws. But this time we're moving to the best-looking planet in the whole universe: Pandora. And what do we need to get rid of, in order to fly there unencumbered and be able to enjoy each tiny detail of a wonderfully reconstructed virtual world?
S is for Shimmering. S is for Stop. S is for silly, which is what I'm being now instead of coming to the point.
Once again, I used to think that this would not be possible. And once again, thanks to your enthusiasm and support, I was proven wrong.
Despite many requests for it, I had been postponing indefinitely the horrible task of fixing TAA or DLSS for our VR supported titles, because it was a huge time sink that would need to be replicated from scratch for each new game. When I did it for the very first mod that was available on this Patreon, I had to spend weeks reverse-engineering the way the game computed and stored the motion vectors, literally down to each individual hair strand in the horses' manes.
It was a bloodbath, and back then I told myself I would never go through such a painful experience again, just to get rid of a few shimmering pixels around nearby objects and NPCs. Unfortunately, without precise motion vectors neither TAA nor DLSS can work properly, so it always seemed to be an all-or-nothing affair where deciding to fix a single game's antialiasing implementation to make it work in stereoscopic 3D would take me more time than modding the game itself, and crush my soul in the process :-(
Then suddenly, a few months ago, I had an idea for an algorithm that could perhaps reconstruct the correct stereoscopic motion vectors (or at least a decent approximation of them) for any game, with no need to reverse-engineer the byzantine ways in which the vectors are computed by the engine, but instead using only the information that the engine already provides to NVIDIA's DLSS libraries.
I didn't announce anything, because I didn't want to give people false hope in case the idea turned out to be a dead end. Also, as they say, ideas are a dime a dozen, and execution is everything. So I patiently set down to work on it, and as usual it took me WAY longer than I had anticipated. Many failed attempts, more sleepless nights than I care to admit, a number of apparently promising intuitions that turned out to be completely wrong. And a plethora of details to take care of, dozens of unexpected corner cases to smooth out.
But in the end, as you perhaps already saw from Brandon's video, the result was actually better than I had dared hope for. I thought there would be some incremental progress in image quality, that maybe I could make the defects a little less annoying; as a matter of fact, sometimes the upscaling is almost as good as native, and I was genuinely surprised at how much missing information it's actually possible to reconstruct without coaxing the game engine into doing any extra work.
Now, before you get carried away, I want to go into some details of what this new DLSS fix can and cannot do, and also give you some hints on how to make it function at the best of its capabilities.
Want the good news or the bad news first?
Yeah, let's start with the really bad stuff. Just like AER v2, this is NVIDIA only. If you have an AMD card, you won't be able to take advantage of the new technique with my mods. I apologize to Team Red, but I'm only one person. I know there are some third-party mods that make it possible to emulate DLSS on AMD cards using FSR, and it's possible they will work with this new tech of mine, but it is completely uncharted territory and you will have to experiment on your own. Also, though I have no affiliation whatsoever with NVIDIA and receive no money from them, I think that their cards are simply the best choice for VR. And basically everyone on the web seems to agree that DLSS is far superior to FSR in terms of image quality. AMD cards just don't have the hardware needed to compute dense optical flow in real time, and there's only so much you can do in software.
Next, the still-bad-but-not-quite-as-much stuff: this only works with DLSS. Games that only support TAA but not DLSS will not see any improvements. That is because, unlike DLSS, ordinary in-engine TAA doesn't have any standardized way of conveying the motion information, so it's not possible for me to make a fix that will work across different engines.
Moving on to the so-so news. The technology is based on a series of approximations, heuristics and a real-time reconstruction of missing data, so it is nowhere perfect. In many cases it works amazing well. But there are other situations where it fails. In particular, distant moving objects will tend to show longer trails than they do in 2D. There will be some shimmering left, especially on sharp contours. Some games may also give less satisfactory results, like for instance titles based on the UE4 engine.
Also, having DLSS work (almost) correctly doesn't mean that we no longer need to use AER. The requirement for the game to render two stereoscopic images instead of just one remains. So the motion artifacts of AER v2 (or the judder of legacy AER) will still be there, completely unchanged.
But there's good news too! First of all, the new tech is essentially free, in the sense that it has no measurable performance hit compared to normal (non stereoscopic) DLSS. And even though there might be moments in the game where it doesn't work perfectly, it is always a big improvement in quality versus the old unfixed TAA or DLSS, especially on large nearby objects like NPC faces and hair.
Moreover, the fix is more or less "universal", in the sense that it is possible for me to implement it on new games with just a reasonable amount of customization. Talking days instead of weeks. Already in this release, the tech will be immediately available for all the titles in our list that offer support for DLSS (including Elden Ring, which has no native support but can do DLSS via a third-party mod). Of course you will also get a checkbox to disable it in case you find any portions of the game on which it doesn't work satisfactorily.
And finally, the super-good news: the quality improvement that you can squeeze out with DL(S)SS is so vast that you can now actually use DLSS not just on the Quality setting, but also on Balanced and all the way down to Performance, despite the huge magnification that VR applies to everything. Together with the FOV optimizations, this means that for the very first time it's becoming possible to play a game in VR with essentially the same image quality that you would get in 2D.
I think the importance of this cannot be overstated. Now maybe the studios and publishers will finally accept that they can offer official VR ports of their games without huge investments, while at the same time keeping the graphics quality at AAA levels.
See you tomorrow for the release!
Comments
Brandon(Post Maester)
Did you update the game to DLSS 4? Did you switch to the newest version of the mod? There was a game update a bit ago that was supposed to improve stability. Are you on the latest GPU driver? Maybe try rolling back a driver if so, unless performance was the same after updating the driver. There have been so many games Luke has done recently that I haven't played them much :( Except for Rebirth and DS2 because those are games I actually very much want to play!! I kinda got taken out of the flow of the story for Avatar and just haven't jumped back in other than for testing. Have you made any changes in Oculus Debug Tool? How are you connecting your Q3? Also double check your game video/display settings against the release post to make sure they are set correctly.
Salvador jimenez
Hi id like to bounce of an issue that I'm not sure how to resolve, below i'll specify hmd, specs and resolution steps: HMD: quest 3 GPU: RTX 4090 CPU: 9800x3d to experience best fidelity with this setup i found myself aiming for 60fps on game resolution (as presented in the REAL Mod overlay) relatively high settings, DLSS Quality (HQ,Q,B,P,UP) around 27.9 -30 PPD 1/3 or 1/2 rate it was working very nice, Pandora looked beautiful as hell and gameplay was quite fluid with just the trade offs i was ok with at the frame rate (FR) generated. now for some reason i've not been able to recreate that experience - my FR are tanking down to 40/30fps and even when switching to DLSS P i literally get no gains in FR, which is weird. i've made sure all background apps are closed, i've played around with settings (minimized RT related settings to low or off) i've adjusted PPD to much lower than comfortable to "provoke" some FR gains and executed a clean RealConfig.bat (deleting previous configs). only effective measure is lowering PPD significantly - not good. another thing to note - when setting the PPD scale to another value the "adapt" button presents the resolution at that PPD, that resolution is quite higher than i would expect (compared to other games and my memory from when setting identical PPD that resulted in the target performance I'm now chasing) i've checked the nvidia app in case there's any Global DSR settings set but it's still set to "OFF". I'm not sure where the increased resolution is stemming from, and I'm not sure why suddenly the performance is tanking (quite curious since the fidelity is significantly lowered and I'm still not able to produce near 60 fps). sorry for the long text, i just wanted to provide details on the matter - just like i would've wanted it. but ask away for clarification. any advice will be appreciated
Goooost
While in the cockpit view of Panams Tank, look is reversed. It's fine in 3rd person, but in side the vehicle I look left and the camera turns right. Outside in 3rd person, the targeting reticle isn't really there either so I have to hope I'm aiming correctly.
Brandon(Post Maester)
It just caused issues for me. I don't see how it would work but a couple people said they have gotten it working. Couldn't get any kind of answer on what settings they used when I asked though. Lossless scaling would affect the monitor image of the game and not the headset image.
RaReLambo21
Do you think this will also work with loss less scaling or no?
