Applications on Android have a very modular architecture; each is, by default, independent of all other applications. Android is Linux-based and inherits the Unix userID model. Each Android application runs under its own userID and launches a new JVM under a new process. If one Android application crashes, it shouldn't affect other applications at all. By the same token, Android applications don't share data or libraries, other than those supplied by Android. The SDK has functions that allow for interprocess communications as designated by the application developer.
The following video from Google gives a good overview of the key areas of the Android application framework for new developers. Note however that it is from 2008, and while the overview still holds good, a couple of things are different in more recent Android versions: most importantly onFreeze() is now called onSaveInstanceState().
Once you've downloaded the Android SDK using the link at the top of this page, and extracted it to somewhere suitable such as your home directory, you need to get your IDE set up for Android development.
Eclipse is the recommended IDE, and it is a fairly heavyweight IDE for Java development that takes a bit of getting used to. It is itself a Java app.
Once you have that, you can download and install the ADT plug-in from within Eclipse itself, by following the instructions here. Make sure you follow all the steps, including those in the Configure the ADT Plugin section lower down on the same page.
The final step to install the Android platform(s) that you're targeting. You do this using the Android AVD manager that comes with the SDK, and which thanks to the ADT plugin is now available as an option in Eclipse's Window menu. Follow the instructions on the android developer site.
You should now be ready to start building Android apps from your desktop, and you can also run them from your desktop using the Android emulator that comes with ADT.
I recommend trying a few of the tutorials on the Android developer site to get to grips with some of the fundamentals of the architecture. The views tutorials are a good start. Below is a screenshot of the 'Hello ListView' tutorial project running in the emulator:
