1) if the jungler comes right now, will I die or burn flash? (Ignore if you know the jungler can't come) this also applies to midland and people with teleports/glob ults and no one at all.
2) if I go back now will the enemy get a gold/exp advantage on me by staying and pushing the wave?
3) if I don't go back now will the enemy be able to achieve a leveragable item advantage on me?
4) do I need to get items in order to achieve a team objective that will be in play by the time I get back?
If the answers to all except number 2 are no then stay in lane and keep farming. If not you should then you should probably back. As you think about those questions you should realize the advantage you can gain if you can force the enemy botane to back at inopportune times and attempt use manipulate the lane to cause them and take advantage of them