In the System.Windows namespace there is an 'Application' object. Your application should make use of this class since it allows for lifetime tracking through events like StartUp and SessionEnding and methods like Run.

We can expand our hello world to:

using System;
using System.Windows;


namespace HelloWorld
{
    public class MyApp : Application
    {
        [STAThread]
        static void Main(string[] args)
        {
            MyApp app = new MyApp();
            app.Startup += app.OnApplicationStart;
            app.Run(args);
        }


        void OnApplicationStart(object sender, StartupEventArgs e)
        {
            Window w = new Window();
            w.Title = "Mark says: Hello World!";
            w.Show();
        }
    }
}

With a little bit of digging around you can find the code that Visual Studio uses for initializing the application:

public partial class MyApp : System.Windows.Application
{
    /// <summary>
    /// InitializeComponent
    /// </summary>
    public void InitializeComponent()
    {
        this.StartupUri = new System.Uri("Window1.xaml", System.UriKind.Relative);
        System.Uri resourceLocater = new System.Uri("myapp.baml", System.UriKind.RelativeOrAbsolute);
        System.Windows.Application.LoadComponent(this, resourceLocater);
    }



    /// <summary>
    /// Application Entry Point.
    /// </summary>
    [System.STAThreadAttribute()]
    public static int Main(string[] args)
    {
        System.Threading.Thread.CurrentThread.SetApartmentState(System.Threading.ApartmentState.STA);
        MyApp app = new MyApp();
        app.InitializeComponent();
        return app.Run(args);
    }
}

As you can see some resources are initialized in the InitializeComponent. By default the VS2005 WinFx project sets up some resources, you find them under the 'Properties' folder. Also an URI is set to track the initial window. This URI is the window that will be started when app.Run(..) is executed.