Leaving out foreign key when creating a postgresql table

I have been using mysql for the past number of years and i have always wondered? Lets say you have several tables and the tables are so interconnected, in an normal schema, you would have foreign keys all over.

What I do when i have to reference another table is to store the id of the table i am referencing and not indicate that its a foreign key.

Must you indicate in a table schema that a field is a foreign key so that you use JOIN keyword later on in my queries or can i still use joins if I understand which columns are foreign keys, they are just not indicated as such?