FIPS or Federal Information Processing Standard presents a set of standards by which information is to be handled. Specifically, the FIPS 140-2 standard sets the security requirements for cryptographic material, and level 2 of that standard, the level that the BIG-IP 8900 adheres to, adds requirements for physical tamper-evidence and role-based authentication to the "key store". The FIPS module that you can purchase with the BIG-IP is a hardware-based security module (HSM) that is a card attached to the motherboard that provides secure storage of cryptographic keys. There are two ways to get new keys into the card: you can import them (which implies that you have a soft copy somewhere), or you can create them there as part of the CSR process to generate new certificates. In either case, once the key is loaded into the HSM, the FIPS certification guarantees a level of protection from extraction attempts. You can, technically, export the keys from the HSM, but only in a proprietary encrypted (and unusable) format. The format is similar to PKCS12.
I also want to point out that the HSM only stores private keys. PKI requires two keys (public and private). The public keys (certificates) are still stored in the file system.