1. Using Files

Written by Jenn Bailey

There are many ways to store data in an Android app. One of the most basic ways is to use files. Similar to other platforms, Android uses a disk-based file system that you can leverage programmatically using the File API. In this chapter, you’ll learn how to use files to persist and retrieve information.

Reading and writing files in Android

Android separates the available storage into two parts, internal and external. These names reflect an earlier time when most devices offered built-in, non-volatile memory (internal storage) and/or a removable storage option like a Micro SD card (external storage). Nowadays, many devices partition the built-in, permanent storage into separate partitions, one for internal and one for external. It should be noted that external storage does not indicate or guarantee that the storage is removable.

There are a few functional differences between internal and external storage:

• Availability: Internal storage is always available. External storage, on the other hand, is not. Some devices will let you mount external storage using a USB connection or other option; this generally makes it removable.

• Accessibility: By default, files stored using the internal storage are only accessible by the app that stored them. Files stored on the external storage system are usually accessible by everything — however, you can make files private.

• Uninstall Behavior: Files saved to the internal storage are removed when the app is uninstalled. Files saved to the external storage are not removed, even when the app is uninstalled. The exception to this rule is if the files are saved in the directory obtained by getExternalFilesDir.

If you’re unsure which method to use, here’s a hint: Use internal storage when you don’t want the user or any other apps to access the files, like important user documents and preferences. Use external storage when you want to allow users or other apps access to the files. This might include images you capture within the app.

Getting started

To get started, you’ll build an app that uses internal storage. Inside this chapter’s folder resources, locate using-files and open projects. Once there, open SimpleNote located inside starter. Wait for the project to sync, download its dependencies and set up the workplace environment. When finished, run the app on a device or in the simulator. For now, you can ignore the warnings in the code.

This app has a simple user interface with two EditTexts, one for the filename and one for a note. There are also three buttons along the bottom: READ, WRITE and DELETE. The user interface looks like this:

The sample for this project includes three sub-packages: model, ui and app.

• model: This package includes a simple data class to represent a single note in Note.kt. NoteRepository.kt contains the declaration of an interface with methods to add, get and delete a Note. ExternalFileRepository, InternalFileRepository and EncryptedFileRepository are classes that implement NoteRepository and override its methods. You’ll be adding code to these classes later.

• ui: This package includes MainActivity.kt. MainActivity implements the OnClick methods for each button. Because the code to read, write, encrypt and delete is abstracted behind NoteRepository and placed into separate classes, the code to handle the button click events remains the same regardless of the type of storage being utilized. This code is dependent on a single repository, and only the concrete type of the repository needs to change.

• app: This package includes Utility.kt, which contains a utility function to produce Toast messages.

Using internal storage

The internal storage, by default, is private to your app. The system creates an internal storage directory for each app and names it with the app’s package name. When you uninstall the app, files saved in the internal storage directory are deleted. If you need files to persist — even after the app is uninstalled — use external storage.

Are you ready to see internal storage in action?

Writing to internal storage

Open MainActivity.kt. Notice the code immediately above onCreate():

private val repo: NoteRepository by lazy { InternalFileRepository(this) }

This is a lazy value. It represents an object of a class that implements NoteRepository. There’s a separate implementation for each storage type demonstrated in this chapter.

The repo is initialized the first time it’s used and will be utilized throughout MainActivity. This includes the button click events that call the add, get and delete methods required by NoteRepository.

In onCreate(), locate btnWrite.setOnClickListener add the following code for the WRITE button’s click event:

// 1
if (edtFileName.text.isNotEmpty()) {
  // 2
  try {
    // 3
    repo.addNote(Note(edtFileName.text.toString(),
        edtNoteText.text.toString()))
  } catch (e: Exception) { // 4
    showToast("File Write Failed")
  }
  // 5
  edtFileName.text.clear()
  edtNoteText.text.clear()
} else { // 6
  showToast("Please provide a Filename")
}

Here’s how it works:

1. Use an if/else statement to ensure the user entered the required information.
2. Put repo.addNote into a try/catch block. Writing a file can fail for different reasons like permissions or trying to use a disk with not enough space available.
3. Call addNote(), passing in a Note that contains the filename and text provided in the EditText fields.
4. If writing the file fails, display a Toast message and write the stacktrace of the error to Logcat.
5. To prepare the interface for the next operation, clear the text from edtFileName and edtNoteText.
6. If the user did not enter a filename, display a toast message within the else block. showToast is a utility function that exists in Utility.kt.

The code for the READ and DELETE button click events are similar to what you added for the WRITE button; these already exist in the sample project.

It’s time for the next step!

Open InternalFileRepository.kt and locate addNote. Then, add the following to the body of the method:

context.openFileOutput(note.fileName, Context.MODE_PRIVATE).use { output ->
  output.write(note.noteText.toByteArray())
}

This code opens the file in fileOutputStream using the Context.MODE_PRIVATE flag; using this flag makes this file private to this app. The FileOutputStream is a Closeable resource so we can manage it using the use Kotlin function. The note’s text is converted to a ByteArray and written to the file.

Run the program. Enter Test.txt for the file name and some text for the note. When you’re ready, tap the WRITE button. If the write is successful, the EditText controls will clear. Otherwise, the stacktrace is printed to Logcat.

Now that you’ve learned how to read, write and delete files on the internal storage, wouldn’t it be nice to see a visual representation of the files in the file system?

Viewing the files in Device File Explorer

In Android Studio, there’s a handy tool named Device File Explorer. This tool allows you to view, copy and delete files that are created by your app. You can also use it to transfer files to and from a device.

Note: A lot of the data on a device is not visible unless the device is rooted. For example, in data/data/, entries corresponding to apps on the device that are not debuggable are not expandable in the Device File Explorer. Much of the data on an emulator is not visible unless it’s an emulator with a standard Android (AOSP) system image. Be sure to enable USB debugging on a connected device.

Open the Device File Explorer by clicking View ▸ Tool Windows ▸ Device File Explorer or by clicking the Device File Explorer tab in the window toolbar.

The Device File Explorer displays the files on your device. Scroll down and locate the data folder. Open data > data > com.raywenderlich.simplenote > files; you’ll see Test.txt and any other files you’ve saved. Files are saved in a directory with the same name as the app’s package name.

Note: The file location depends on the device; some manufacturers tweak the file system, so your app directory might not be where you expect it. If that’s the case, you can locate the folder using the app’s package name as this never changes.

The files stored on the external storage are located in sdcard/data/app_name/.

At the top of the Device File Explorer, there’s a drop-down you can use to select the device or emulator. After making your selection, the files appear in the main window. You can expand the directories by clicking the triangle to the left of the directory name.

Right-click the filename, and a menu pops-up that allows you to perform different operations on the file.

1. Open lets you open the file in Android Studio.
2. Save As… lets you save the file to your file system.
3. Delete allows you to delete the file.
4. Synchronize synchronizes the file system if it’s changed since the last run of the app.
5. Copy Path copies the path of the file to the clipboard.

Now that you know how to write a file to internal storage and use the Device File Explorer to see what files are available, it’s time to learn how to make your app read them.

Reading from internal storage

Replace the current return statement in the getNote in InternalFileRepository with the current code:

// 1
val note = Note(fileName, "")
// 2
context.openFileInput(fileName).use { stream ->
  // 3
  val text = stream.bufferedReader().use {
    it.readText()
  }
  // 4
  note.noteText = text
}
// 5
return note

Here’s how it works:

1. Declare a Note, passing in a fileName and an empty string so that a valid object gets returned from this function even if the read operation fails.
2. Open and consume the FileInputStream with use.
3. Open a BufferedReader with use so that you can efficiently read the file.
4. Assign the text that was read to the file to note.noteText.
5. Return note.

Run the app and enter the name of a file you previously saved and then tap READ. The note’s text displays in the app. And that’s it! Your app can now write and read notes.

Up next, you’ll write the code to delete a file.

Deleting a file from internal storage

Replace the existing return statement for thedeleteNote function in InternalFileRepository with the following line of code:

return noteFile(fileName).delete()

The delete function of the File class returns a Boolean value to indicate whether the delete operation was successful; this function returns that value to where it was called in MainActivity.

Build and run the app. Delete a file by tapping the DELETE button; you’ll see the appropriate message.

To confirm the file was deleted, use the Device File Explorer or try to delete the file a second time. If everything worked as expected, you’ll see a message that indicates the file could not be deleted (because it’s already gone).

Internal storage is great for storing private data in an app. But what if you want to store data temporarily? To do this, you can use Internal Cache Files.

Internal cache files

Each app has a special and private cache directory to store temporary files. Android may delete these files when the device is low on internal storage space, so it’s not safe to store anything other than temporary files in this space. There’s also no guarantee that Android will delete these files for you, so you must maintain this directory yourself.

To write to the internal cache directory, use createTempFile as shown in the following example:

File.createTempFile(filename, null, context.cacheDir)

A good use case for temporary files is when you’re uploading images to a server. You may not need the image persisted on the device, but you still need to upload some file. In this case, a temporary file works well. You’d store the image in this temp file, upload it, and then delete the file upon completion.

Now that you have utilized the important features of internal storage, it’s time to look at how to store files on External Storage.

Using external storage

External storage is appropriate for user data that you want to make accessible to the user or other apps. Files saved on the external storage system are not deleted — even when the app is uninstalled. The external storage is made up of standard public directories. Files saved to the external storage are world-readable, by default, and can be modified by enabling mass storage and transferring the files to the computer via USB.

External storage is not guaranteed to be accessible at all times; sometimes it exists on a physically removable SD card. Before attempting to access a file in external storage, you must check for the availability of the external storage directories, as well as the files. You can also store files in a location on the external storage system, where they will be deleted by the system when the user uninstalls the app.

Now that you know a bit more about external storage possibilities, it’s time to modify the app to use external rather than internal storage. You’ll start by adding the necessary permissions to the manifest.

Adding permissions in the manifest

To use external storage, you must first add the correct permission to the manifest. If you wish to only read external files, use the READ_EXTERNAL_FILE permission.

<uses-permission 
  android:name="android.permission.READ_EXTERNAL_STORAGE" />

If you want to both read and write, use the WRITE_EXTERNAL_STORAGE permission.

<uses-permission 
  android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

Beginning with API level 19, reading or writing files in your app’s private external storage directory does not require the above permissions. If your app supports Android API level 18 or lower, and you’re saving data to the private external directory only, you should declare that the permission is requested only on the lower versions of Android by adding the maxSdkVersion attribute:

<uses-permission 
  android:name="android.permission.WRITE_EXTERNAL_STORAGE"
  android:maxSdkVersion="18" />

In this app, you’ll be reading and writing to the external storage, so add the WRITE_EXTERNAL_STORAGE permission inside the <manifest> element in AndroidManifest.xml.

Writing the notes to external storage

Now that the correct permissions are in place, it’s time to write the file to the external storage. Open ExternalFileRepository.kt and add the following code to addNote:

// 1
if (isExternalStorageWritable()) {
  // 2
  FileOutputStream(noteFile(note.fileName)).use { output ->
    output.write(note.noteText.toByteArray())
  }
}

Here’s how it works:

1. Check to see if the external storage is available.
2. Open a FileOutputStream with use.
3. Write note.noteText to the file.

Next, in MainActivity.kt, change the instance of the NoteRepository you’re initializing to the following:

private val repo: NoteRepository by lazy { ExternalFileRepository(this) }

Finally, run the program to write a file to the external storage. To view the file using the Device File Explorer, look in sdcard/data, within the app’s package name folder.

You’ll implement the app’s read function in the next step.

Reading from external storage

Replace the existing return statement of the getNote function with the following code:

val note = Note(fileName, "")
// 1
if (isExternalStorageReadable()) {
  // 2
  FileInputStream(noteFile(fileName)).use { stream ->
    // 4
    val text = stream.bufferedReader().use {
      it.readText()
    }
    // 5
    note.noteText = text
  }
}
return note

For the most part, the procedure here is the same as reading from the internal storage but with a small difference. Here’s how it works:

1. Ensure the external storage is readable.
2. Open and consume the FileInputStream, with use, as you did before.
3. Open a BufferedReader with use so that you can efficiently read the file.
4. Assign the text that was read to the file to note.noteText and return the note.

The above code also relies on the following two functions:

fun isExternalStorageWritable(): Boolean {
  return Environment.getExternalStorageState() == Environment.MEDIA_MOUNTED
}

fun isExternalStorageReadable(): Boolean {
  return Environment.getExternalStorageState() in
      setOf(Environment.MEDIA_MOUNTED, Environment.MEDIA_MOUNTED_READ_ONLY)
}

The isExternalStorageWritable function determines if the external storage is mounted and ready for read/write operations, whereas isExternalStorageReadable determines only if the storage is ready for reading.

After writing and reading files, you’re ready to add the capability to delete a file from external storage.

Deleting a file from external storage

Insert the following code into deleteNote in the ExternalFileRepository.kt file:

return isExternalStorageWritable() && noteFile(fileName).delete()

The first part of the condition checks if the external storage can be written to or altered; the second part, if the first condition is true, returns the result of deleting the file. This way, you can be sure that the file will be deleted only if you can manipulate external storage.

Securing user data with a password

Security is important for the credibility of your app, especially when it comes to securing users’ private data. Storing data on external storage allows the data to be visible to other apps. That’s why, as a general rule, it’s advised to avoid using external storage. Or at least doing so, without a strong security system and encryption.

You can prevent users from installing apps on external storage altogether. Even when an app-only uses internal storage, having it installed on external storage could allow user to copy your app binary and data. To prevent users from installing the app on external storage you can add android:installLocation with a value of internalOnly to the manifest file.

Another best practice you can use to enhance your app’s security is to prevent the contents of the app’s private data directory from being downloaded with adb backup. You do this by setting the android:allowBackup attribute to false in the manifest file. This overrides the default value of true.

Although these are good strategies to use to secure your app’s files, the user can undermine them if the device is compromised or rooted. Even the built-in disk encryption is ineffective if the device is not secured with a lock screen.

One way to secure your data beyond the best practices listed above is to encrypt the files before writing them to the external file system with a user-provided password.

Using AES and Password-Based Key Derivation

The recommended standard to encrypt data with a given key is the AES (Advanced Encryption Standard). In this example, you’ll use the same key to encrypt and decrypt data - known as symmetric encryption. The preferred length of the key is 256 bits for sensitive data.

It’s not realistic to rely on the user to select a strong or unique password, that’s why it’s never recommended to use passwords directly to encrypt the data. Instead, produce a key based on the user’s password using Password-Based Key Derivation Function or PBKDF2.

PBKDF2 produces a key from a password by hashing it over many times with salt. This creates a key of a sufficient length and complexity, and the derived key will be unique even if two or more users in the system used the same password.

In this example, a String that represents the user’s password has been hardcoded at the top of EncryptedFileRepository.kt: val passwordString = "Swordfish".

Find encrypt and add the code inside the empty try block, replacing the TODO:

// 1
val random = SecureRandom()
// 2
val salt = ByteArray(256)
// 3
random.nextBytes(salt)

Here’s how it works:

1. Generate a random value using the SecureRandom class. This guarantees the output is difficult to predict as SecureRandom is a cryptograpically strong random number generator.
2. Create a ByteArray of 256 bytes to store the salt.
3. Pass the salt to nextBytes which will fill the array with 256 random bytes.

Next, add the following code to the random.nextBytes call to salt the password.

// 4
val passwordChar = passwordString.toCharArray() 
// 5
val pbKeySpec = PBEKeySpec(passwordChar, salt, 1324, 256) //1324 iterations
// 6
val secretKeyFactory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1")
// 7
val keyBytes = secretKeyFactory.generateSecret(pbKeySpec).encoded
// 8
val keySpec = SecretKeySpec(keyBytes, "AES")

Here’s how it works:

4. Convert the password into a character array.

5. Pass the password in char[] form, along with the salt, to PBEKeySpec, as well as the number of iterations, 1324, and the size of the key, 256. Increasing the number of iterations also increases the time it would take to operate on a set of keys during a brute-force attack.

6. Generate an instance of a SecretKeyFactory using PBKDF2WtihHmacSHA1.

7. Pass the pbKeySpec to the secretKeyFactor.generateSecret method and assign the encoded property which returns the key in bytes.

8. Finally, keySpec is produced to use when you initialize the Cipher.

All of these steps seem a bit complex, but it’s the kind of thing that’s always the same to use; only the data changes. There are still a few steps to do, so let’s get to it!

Using an initialization vector

The recommended mode of encryption when using AES is the cipher block chaining, or CBC. This mode takes each next unencrypted block of data and uses the XOR operation with the previous encrypted block. One problem with this procedure is that the first block is less unique as subsequent blocks. If one encrypted message started the same as another message, the beginning blocks of the two messages would be the same. This would help an attacker to find out what the message(s) are. To solve this problem, you’ll create an initialization vector or an IV.

An IV is a block of random bytes that are XOR’d with the leading block of the data. All subsequent blocks are dependent on the previous block, so using an IV uniquely encrypts the entire message.

Inside encrypt, immediately below where you created the keySpec, add the following code to create an IV:

// 9
val ivRandom = SecureRandom()
// 10
val iv = ByteArray(16)
// 11
ivRandom.nextBytes(iv)
// 12
val ivSpec = IvParameterSpec(iv)

Here’s what’s happening:

9. Create a new SecureRandom object so that you’re not using a cached, seeded instance.

10. Create a byte array with a size of 16 to store the IV.

11. Populate iv with random bytes.

12. Create the IvParameterSpec with the random iv so that you can use it when performing the encryption.

Now that you have created keySpec and ivSpec it’s time to use them to encrypt a note. In the next step, add the code to create a Cipher and encrypt the data to be stored and place the data, the salt and the iv into HashMap map to be returned by the method.

// 13
val cipher = Cipher.getInstance("AES/CBC/PKCS7Padding")
cipher.init(Cipher.ENCRYPT_MODE, keySpec, ivSpec)
// 14
val encrypted = cipher.doFinal(plainTextBytes)
// 15
map["salt"] = salt
map["iv"] = iv
map["encrypted"] = encrypted

Here’s the explanation:

13. Create and initialize the Cipher using AES/CBC/PKCS7Padding. This specifies AES encryption with cypher block chaining. PKCS7 refers to an established standard for padding data that doesn’t fit into the specified block size.
14. Encrypt the bytes of the data with the cipher.
15. Place the salt, iv, and encrypted bytes into a HashMap.

The IV and salt are considered public and can be stored with your data. However, make sure they are not reused or sequentially incremented.

Now the encrypt method can be used to encrypt the text of a note before that text is written to a file. To finish off the encryption of the file before storing it, insert the following code into addNote:

if (isExternalStorageWritable()) {
  ObjectOutputStream(noteFileOutputStream(note.fileName)).use { output ->
    output.writeObject(encrypt(note.noteText.toByteArray()))
  }
}

The code above creates an ObjectOutputStream with use and utilizes it to write the encrypted message out to the file. Before you run the app again, you have to change the instance of the NoteRepository in MainActivity.kt to this:

private val repo: NoteRepository by lazy { EncryptedFileRepository(this) }

Run the app and use it to write an encrypted file. If you then find and open the file with Device File Explorer, you’ll see that it’s filled with unreadable data because the file is encrypted.

Now you must add a mechanism to decrypt the file.

Decrypting the file

Locate decrypt and add the following code:

private fun decrypt(map: HashMap<String, ByteArray>): ByteArray? {
  var decrypted: ByteArray? = null
  try {
    // 1
    val salt = map["salt"]
    val iv = map["iv"]
    val encrypted = map["encrypted"]
    // 2 
    val passwordChar = passwordString.toCharArray()
    val pbKeySpec = PBEKeySpec(passwordChar, salt, 1324, 256)
    val secretKeyFactory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1")
    val keyBytes = secretKeyFactory.generateSecret(pbKeySpec).encoded
    val keySpec = SecretKeySpec(keyBytes, "AES")
    // 3 
    val cipher = Cipher.getInstance("AES/CBC/PKCS7Padding")
    val ivSpec = IvParameterSpec(iv)
    cipher.init(Cipher.DECRYPT_MODE, keySpec, ivSpec)
    decrypted = cipher.doFinal(encrypted)
  } catch (e: Exception) {
    Log.e("SIMPLENOTE", "decryption exception", e)
  }
  // 4
  return decrypted
}

Here’s what’s happening:

1. Retrieve the salt, iv and encrypted fields from the HashMap.
2. Regenerate the key from the password.
3. Decrypt the encrypted data.
4. Return the decrypted data.

Call the decrypt function after reading the file in getNote.

val note = Note(fileName, "")
if (isExternalStorageReadable()) {
  // 1
  ObjectInputStream(noteFileInputStream(note.fileName)).use { stream ->
    // 2
    val mapFromFile = stream.readObject() as HashMap<String, ByteArray>
    // 3
    val decrypted = decrypt(mapFromFile)
    if (decrypted != null) {
      note.noteText = String(decrypted)
    }
  }
}
return note

Here’s how it works:

1. Open an ObjectInputStream with use for reading data.
2. Read the data and store it into the HashMap.
3. Decrypt the file with decrypt; if it was successful, assign the decrypt text to note.noteText.

Build and run the code now. Notice the weird, encrypted, obfuscated data is now decrypted and understandable again! :]

Understanding Parcelization and Serialization

In computer science, ** marshaling** or marshaling is the process of transforming an object into a format that is suitable for transmission or storage. Android apps often transfer data from one activity to another, where Parcelization and Serialization are the means of marshaling objects.

By default — and because your app utilized ObjectOutputStream to write the file — the encrypted data, iv and salt values were all Serialized when written to the file.

Serializable

Serializable is a standard Java tagging interface. This interface has no operation but it can be used to define the corresponding type as serializable. An object is serializable when it can be converted into an array of bytes and vice versa. If an object is Serializable it can also be written into a file or read from a file. Serialization is not very easy though. When you restore an object you still need a compatible version of the class used during serialization. Implementing the Serializable interface isn’t enough - some properties are implicitly not serializable. Some examples are Thread or InputStream. The Serialization process implies the use of reflection.

Reflection is the ability for an object to know things about itself. Therefore, using serializable can result in a lot of garbage collection which then translates to poor performance and battery drain. Fortunately, there’s another way to marshal objects in Android.

Parcelable

Parcelable, like Serializable, is an interface. However, unlike Serializable, Parcelable is part of the Android SDK, not the standard Java interface. Because it’s designed not to use reflection, it requires the developer to be explicit about the marshaling process, making it more tedious to use. However, despite this complication, using Parcelable can result in better app performance — although the gain in performance is usually imperceptible to the user.

Parcelable is often used when passing data between activities in a Bundle.

Note: The Parcelable API is not for general purposes. It’s designed to be a high-performance IPC transport. It’s not appropriate to place Parcel data into persistent storage because any change to the underlying implementation of the data can render the old data unreadable.

Key points

• Files are a quick way to persist data in Android.
• The internal storage is a great place to store files that are specific and private to your app.
• Use the internal cache to store temporary files.
• Use external storage to store files that you want users or other apps to have access to.
• External files are persistent, even after the app is uninstalled.
• Internal files are deleted when the app is uninstalled.
• To write files to the external storage, the correct permissions must be set in the manifest.
• Because the external storage is not secure, it’s a good idea to use AES encryption with a password-based generated key.
• As a general security practice, apps that store data files should not be allowed to be installed on external storage.
• Serializable and Parcelable are ways of marshaling data for transport or storage.
• Do not use Parcelable to store persistent data; if the Parcel is changed, the old data will not be read properly.

Where to go from here?

File encryption — and encryption in general — is a broad topic. To learn more about it, read the tutorial located at https://www.raywenderlich.com/778533-encryption-tutorial-for-android-getting-started. To dig deeper into file management on Android, also read the official documentation available on the Android Developer site at https://developer.android.com/guide/topics/data/data-storage.