Brightness Changing automatically

[kispintyi]

For some reason Time settings doesn't respect my manually set brightness level and will randomly decide to pump it up to 40% when I set it manually to 30% due to me having a setting for day and night brightness levels. 0% night and 40% day. Why is this and how can I make it where it will always go to the right brightness level at the right time but still respects my manually selected brightness?