Well, it is. Kind of. Almost. There are only two things you have to remember about.

The verb

Where English uses the verb “to be” in such constructions, Polish prefers verbs that actually have a “real” meaning – ones that say something about the subject’s position in space. You can use “to be”, of course, but there will almost always be some alternative that actually sounds more natural. For example:

Na stole jest gazeta.
There is a newspaper on the table.
A grammatically correct, meaningful sentence, but it sounds like there’s something… weird with it.
Na stole leży gazeta.
There *lies* a newspaper on the table.
What a native speaker would say.

The verb leżeć – to lie – is used to talk about objects that are flat and, well, can lie. For example:

Na podłodze lężą książki.
There are books on the floor.

W szufladzie leży złoty pierścionek.
There is a golden ring in the drawer.

W pudełku leży nowy procesor.
There is a new processor in the box.

W szafie leżą stare szmaty.
There are old rags in the wardrobe.

Another useful verb is stać – to stand. You can use it to talk about things that are more vertical:

W lodówce stoi butelka mleka.
There’s a bottle of milk in the fridge.

Na stole stoi komputer.
There’s a computer on the table.

W szafie stoją buty.
There are shoes in the wardrobe.

Na podłodze stoją trzy walizki.
There are three suitcases on the floor.

The word order

One thing you’ve surely noticed by now is the different word order: where the English sentence starts with “there is” and ends with a prepositional phrase – stating where our subject is located – the Polish one opens with a prepositional phrase and puts the subject in the end.

But, you might think, isn’t the Polish word order flexible? Isn’t it, like, the FIRST thing anyone learns, that Polish enables you to toss words around like kittens do with balls of yarn? Well, yes. In fact, the sentence Gazeta leży na stole is absolutely correct. There’s only one small caveat. It doesn’t imply exactly the same thing.

What it really means is: The newspaper is on the table.

See? By moving the subject to the beginning of the sentence, we did the equivalent of sticking a definite article to it. Polish doesn’t have articles, so it uses this maneuver instead. Compare:

Na stole leży gazeta.
There’s a newspaper on the table.
I look at the table and describe it – oh, there’s some newspaper on it! I haven’t seen it before, and I don’t really care about it.

Gazeta leży na stole.
The newspaper is on the table.
I know what newspaper we’re talking about – the one I’ve been searching for all day.

So, if you’re talking about where something familiar is located – something that you would precede with “the” in English – you put it in the beginning of the sentence. And if you’re just looking around and noticing various things lying around, you move them to the end. Once you get used to it, it makes sense. Actually, I could go on and on about how it makes sense by using words like Theme, Rheme and topicalization, but I’ll keep that piece of grammar porn to myself. You don’t want to know it anyway.