Go to the product and click "Edit Product" or
Go to your Dashboard and go to Dashboard->WooCommerce->Products
Scroll down until you see the tab "Inventory", click the link.
Go to "Stock Status", click the dropdown arrow to reveal the options and select "Out of Stock"
Click "Update" to save the changes.