Depends on how complex you want to go... Go all in, like TabbyBoy, you best invest in a hardware Wallet like Ledger Nano S, or even better, a Ledger Blue. Safest and most robust. Then you need to purchase a cryptocurrency at a broker, and have it transferred to your corresponding Wallet-address on the Ledger. To exchange between currencies, a very popular site is Binance.com.
If you want to dip your toe in the water and just get a feeling for it, you could conveniently open an account at Blockchain.info or Coinbase.com. There you get a Wallet connected to your account (including 2-factor authentication) and both have a smartphone/tablet app to access your Wallet on-the-go. Blockchain.info currently supports Bitcoin (BTC), Bitcoin Cash (BCH) and Ethereum (ETH). On Coinbase.com you get the same currencies, and additionally also Litecoin (LTC).
Here is an overview of Wallets with features, supported currencies and user-reviews: https://www.cryptocompare.com/wallets/#/overview