Safari "Switch Between Tab Groups" keyboard shortcut doesn't work properly.

I'm using tab groups to categorize my safari tabs all the time. But it's always a headache for me to change the tab group from top menu bar on safari therefore I've searched for a shortcut. The keyboard shortcuts for "Go to Previous Tab Group" and "Go to Next Tab Group" are "cmd+shift+up" and "cmd+shift+down" according to Apple's default system. But whenever I try to use the key combination above, safari always scrolls down on the webpage instead of switching between the tab groups.

At first, I thought it's an issue and tried to change the shortcut on the "system settings>keyboard>shortcuts" but I then realized that problem is not with the shortcut itself. It's actually working but only on pages that doesn't have scrolling such as safari home page or Google.

According to my understanding if there's nothing to scroll, keyboard shortcut works just fine, otherwise shortcut scrolls down on page instead of switching between tab groups.

How can I fix it and change the combination above as always switch between tab groups instead of scroll down? I guess they intersect&conflict with each other.