It could be related to the charging brick you are using being rather low wattage, try using a more powerful charging brick or using a charging brick from a more well-known brand (since poorly made 3rd party charging bricks may cause issues) and see if that fixes your problem.
Also if you are directly plugging your phone into a computer or laptop's USB port then that might be your reason as the power delivered by those ports tends to be rather low.