![]() |
![]() |
![]() |
![]() |
總覽
本教學課程示範了 tfio.genome
套件,此套件提供常用的基因體學 IO 功能,也就是讀取數種基因體學檔案格式,並提供一些用於準備資料的常用作業 (例如,單次有效編碼或將 Phred 品質剖析為機率)。
此套件使用 Google Nucleus 程式庫來提供部分核心功能。
設定
try:
%tensorflow_version 2.x
except Exception:
pass
!pip install -q tensorflow-io
import tensorflow_io as tfio
import tensorflow as tf
FASTQ 資料
FASTQ 是一種常見的基因體學檔案格式,除了序列資訊外,也會儲存鹼基品質資訊。
首先,我們先下載範例 fastq
檔案。
# Download some sample data:
curl -OL https://raw.githubusercontent.com/tensorflow/io/master/tests/test_genome/test.fastq
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 407 100 407 0 0 2035 0 --:--:-- --:--:-- --:--:-- 2035
讀取 FASTQ 資料
現在,我們使用 tfio.genome.read_fastq
讀取此檔案 (請注意,tf.data
API 即將推出)。
fastq_data = tfio.genome.read_fastq(filename="test.fastq")
print(fastq_data.sequences)
print(fastq_data.raw_quality)
tf.Tensor( [b'GATTACA' b'CGTTAGCGCAGGGGGCATCTTCACACTGGTGACAGGTAACCGCCGTAGTAAAGGTTCCGCCTTTCACT' b'CGGCTGGTCAGGCTGACATCGCCGCCGGCCTGCAGCGAGCCGCTGC' b'CGG'], shape=(4,), dtype=string) tf.Tensor( [b'BB>B@FA' b'AAAAABF@BBBDGGGG?FFGFGHBFBFBFABBBHGGGFHHCEFGGGGG?FGFFHEDG3EFGGGHEGHG' b'FAFAF;F/9;.:/;999B/9A.DFFF;-->.AAB/FC;9-@-=;=.' b'FAD'], shape=(4,), dtype=string)
如您所見,傳回的 fastq_data
具有 fastq_data.sequences
,此為 fastq 檔案中所有序列的字串張量 (每個序列的大小可能不同),以及 fastq_data.raw_quality
,其中包含 Phred 編碼品質資訊,說明序列中每個鹼基讀取的品質。
品質
如果您有興趣,可以使用輔助運算子將此品質資訊轉換為機率。
quality = tfio.genome.phred_sequences_to_probability(fastq_data.raw_quality)
print(quality.shape)
print(quality.row_lengths().numpy())
print(quality)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py:574: calling map_fn_v2 (from tensorflow.python.ops.map_fn) with dtype is deprecated and will be removed in a future version. Instructions for updating: Use fn_output_signature instead (4, None, 1) [ 7 68 46 3] <tf.RaggedTensor [[[0.0005011872854083776], [0.0005011872854083776], [0.0012589251855388284], [0.0005011872854083776], [0.0007943279924802482], [0.00019952621369156986], [0.0006309572490863502]], [[0.0006309572490863502], [0.0006309572490863502], [0.0006309572490863502], [0.0006309572490863502], [0.0006309572490863502], [0.0005011872854083776], [0.00019952621369156986], [0.0007943279924802482], [0.0005011872854083776], [0.0005011872854083776], [0.0005011872854083776], [0.0003162277571391314], [0.0001584893325343728], [0.0001584893325343728], [0.0001584893325343728], [0.0001584893325343728], [0.0010000000474974513], [0.00019952621369156986], [0.00019952621369156986], [0.0001584893325343728], [0.00019952621369156986], [0.0001584893325343728], [0.00012589251855388284], [0.0005011872854083776], [0.00019952621369156986], [0.0005011872854083776], [0.00019952621369156986], [0.0005011872854083776], [0.00019952621369156986], [0.0006309572490863502], [0.0005011872854083776], [0.0005011872854083776], [0.0005011872854083776], [0.00012589251855388284], [0.0001584893325343728], [0.0001584893325343728], [0.0001584893325343728], [0.00019952621369156986], [0.00012589251855388284], [0.00012589251855388284], [0.0003981070767622441], [0.0002511885541025549], [0.00019952621369156986], [0.0001584893325343728], [0.0001584893325343728], [0.0001584893325343728], [0.0001584893325343728], [0.0001584893325343728], [0.0010000000474974513], [0.00019952621369156986], [0.0001584893325343728], [0.00019952621369156986], [0.00019952621369156986], [0.00012589251855388284], [0.0002511885541025549], [0.0003162277571391314], [0.0001584893325343728], [0.015848929062485695], [0.0002511885541025549], [0.00019952621369156986], [0.0001584893325343728], [0.0001584893325343728], [0.0001584893325343728], [0.00012589251855388284], [0.0002511885541025549], [0.0001584893325343728], [0.00012589251855388284], [0.0001584893325343728]], [[0.00019952621369156986], [0.0006309572490863502], [0.00019952621369156986], [0.0006309572490863502], [0.00019952621369156986], [0.002511885715648532], [0.00019952621369156986], [0.03981072083115578], [0.003981071058660746], [0.002511885715648532], [0.050118714570999146], [0.003162277629598975], [0.03981072083115578], [0.002511885715648532], [0.003981071058660746], [0.003981071058660746], [0.003981071058660746], [0.0005011872854083776], [0.03981072083115578], [0.003981071058660746], [0.0006309572490863502], [0.050118714570999146], [0.0003162277571391314], [0.00019952621369156986], [0.00019952621369156986], [0.00019952621369156986], [0.002511885715648532], [0.06309572607278824], [0.06309572607278824], [0.0012589251855388284], [0.050118714570999146], [0.0006309572490863502], [0.0006309572490863502], [0.0005011872854083776], [0.03981072083115578], [0.00019952621369156986], [0.0003981070767622441], [0.002511885715648532], [0.003981071058660746], [0.06309572607278824], [0.0007943279924802482], [0.06309572607278824], [0.001584893325343728], [0.002511885715648532], [0.001584893325343728], [0.050118714570999146]], [[0.00019952621369156986], [0.0006309572490863502], [0.0003162277571391314]]]>
單次有效編碼
您可能也想要使用單次有效編碼器來編碼基因組序列資料 (其中包含 A
T
C
G
鹼基)。有一個內建運算可以用於協助您完成這項作業。
one_hot = tfio.genome.sequences_to_onehot(fastq_data.sequences)
print(one_hot)
print(one_hot.shape)
<tf.RaggedTensor [[[0, 0, 1, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 0, 0, 1], [1, 0, 0, 0], [0, 1, 0, 0], [1, 0, 0, 0]], [[0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1], [0, 0, 0, 1], [1, 0, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 0, 1], [0, 1, 0, 0], [1, 0, 0, 0], [0, 1, 0, 0], [1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 0, 1], [0, 0, 1, 0], [1, 0, 0, 0], [0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 0, 1], [1, 0, 0, 0], [1, 0, 0, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1], [1, 0, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1], [1, 0, 0, 0], [1, 0, 0, 0], [1, 0, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 0, 1], [0, 0, 0, 1], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 0, 1], [0, 0, 0, 1], [0, 1, 0, 0], [1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 0, 1]], [[0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 0, 1], [0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [1, 0, 0, 0], [0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 1, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 1, 0], [1, 0, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 1, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 1, 0, 0]], [[0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 1, 0]]]> (4, None, 4)
print(tfio.genome.sequences_to_onehot.__doc__)
Convert DNA sequences into a one hot nucleotide encoding. Each nucleotide in each sequence is mapped as follows: A -> [1, 0, 0, 0] C -> [0, 1, 0, 0] G -> [0 ,0 ,1, 0] T -> [0, 0, 0, 1] If for some reason a non (A, T, C, G) character exists in the string, it is currently mapped to a error one hot encoding [1, 1, 1, 1]. Args: sequences: A tf.string tensor where each string represents a DNA sequence Returns: tf.RaggedTensor: The output sequences with nucleotides one hot encoded.