The buffer delays the data being sent to the DAC for a fraction of a second. This gives the DAC time to change configuration when a new sample rate is presented to it it e.g. changing from 16/44.1 to 24/192. If it was not there the DAC would not have time to mute and you would hear noises during the change.
Exactly the same data is presented to the DAC before processing, just a fraction of a second later than when the buffer is off.
Therefore unless all of your source material ( including that streamed) is of the same sample rate you should leave the buffer switched on if you don’t want to get noises (and lose roughly the first second or so of the music ) whenever a new sample rate is encountered.