I am having issues with my stream audio having a jerking effect every 5-7 secs in my stream

I am having issues with the audio in my stream to youtube. I get a jerking sound every 5-7 seconds in the stream a sort of flicker in the audio. I have isolated it to the encoder because when I record audio normally via my camera through capture card I dont get this effect. So it has to do with my encoder setting or some bug. Any help would be appreciated.