After some investigations i ended up in a bug report stating issues with processes of xdg-desktop-portal spawned by sddm too soon.
https://bugs.kde.org/show_bug.cgi?id=458865
near the end there's a link to a commit, where its shown an env variable to disable qt from spawning desktop portals, QT_NO_XDG_DESKTOP_PORTAL. This, as far as I understood, was made to prevent the qt part of sddm to set the xcursor before all the rest of the user environment was completely loaded.
So i exported this var in my /etc/environment and rebooted and now I only have desktop-portals under my user.
They also seems to work, at least from firefox i can open my downloads in dolphin just fine. I did notice a slow startup though.