01s0201NoSQL在腾讯应用实践 - 吴悦

NoSQL:»ÃU HË 5d Ö°éjoywu@tencent.com \,ét.qq.com/iwuyue ¤�¢šc Z|u� • dGY|� •  50W+Wh� ~3Hf� •  $)`� 80%eu� Z|J�� •  2��� •  4KB+� Qzone¢œ™ Ð5000 ƒÄZ ’AÐ5 Žw Feeds � vv� M<� Sns Game¢�3 ."��� !u� OK…l� F-H8� dW/S � �dGB3 � �OAPP¢ß” •  LAMPæLinux+Apache+Mysql+Phpç • =zB0�• C‚�{� • �QG� •  IDCG� • †'&� • �:y� ?q� 0}� B5�x[� UM�­ • File3System3<3TFS �g�e_@� • NoSQL<3TDB/TSSD �QZONE�ea^� • NoSQL–3CMEM�SNS3Gamee %� • SQL3Cluster3–3CDB�j�LAPPe‚X�� NoSQL-TDB •  07R¥- •  ˜Ø²Y¯Ÿ •  _ÔϨ TDB-±´Œ‡ •  C.H.M³… HAND TDB Library��� ��� ��� Set Get Del �� Master CELL HAND HAND CELL CELL CELL CELL CELL � � ˜Ø²Y¯Ÿ-@½ÕÍªÛ •  @½ – ½¦���)0À¢½ªÛ – qj&256TB •  O½ –  256MB – @½:TDB�×¾ $ ?�O½ -s� 2s� 2s� 2s� 2s� 2s� ƒ<0� ƒ<1� ƒ<2� ƒ<15� Hash3r(N13 W 2s�s�Vr(� S€,P+ I3 HEADER � +�)� TAIL � ,P+� \�+� ˜Ø²Y¯Ÿ-O½Å •  O½ – $e16�page – ’�page~��hashv� •  ¿Å>Direct IOÇ – áMU¿Å>¢œG�· •  « µE – �ºF�¤�OS¢X – 0o' µE@O – o' ÔV HEADER � +�)� TAIL � ,P+� \�+� _ÔpÏ •  �16MB PAGE +�ѼpÏ 3. � ƒ � � � � 2 s � Master� CELL � CELL �Trsf � t† • ²YE� –  Qzone 5000 ƒÄZ • �´B‰ÅÂß500*32G • @½ÅÂß500M • ª– – <€zª–èªÛ% ž&85% •  IO –  80%et;�TIO NoSQL - TMEM •  09R¥- •  ã`º •  �E0à` Memory œ` •  [ã¢Ý‚Ç `º – +‚300W + IO/PS •  }B`E� – m¡4èul�B TMEM-ã`ºn�+H|>� •  KCCDèÁ"n�º( •  TFSè�ul—“+Ù� 1G# …# 1G#KC C D 1G# …# 1G#KC C D 1G# …# 1G#KC C D 1G# …# 1G#KC C D port3 …# port# K C C D port3 …# port# K C C D TFS# NoSQL-TSSD •  10R¥- •  ݂ � SSD œ` •  qj[ã¢Ç.IOPS –  300GB SSD£ – ݂Ç.4KBè0Î30K/S •  ݂ ©G`7N6�� –  �s@ –  300GB¢SSD£ • âT �é300GB*3000 = 900TB • ݂ �é æ300GB*3000çæ4KB/512KBç = 7TB TSSD-݂ÌâT •  PAGE-MAPPINGh •  8r®• „7���/o��� 512KB� 512KB� 512KB� 512KB�512KB�512KB� PAGE3M APPIN G� 512KB� 512KB� 512KB� 512KB�512KB�512KB� 512KB� 512KB� 512KB� U*��� 6�]� ic� ܑO³ SATA (TFS) j�* @ØÇèw= E�100T+ ¤�èC�è Ö�趣 SAS (TDB) j�* @ØÇPØ E�10+T Qzone { ^ SSD (TSSD) j� |SE @ØÇ@Ø E�1+T �b�]ä … MEM (TMEM) j� |SE @ØÇ@Ø E�10+G !›;ä ¹„�9ä … ‹lIOÆÚLVÓiE� •  ®1§�ÉÒ2¢IOÆÚLV ��� � ($) ��(GB) IOPS GB IOPS GB��($) SATA 200 2000 100 1/20 0.10 SAS 370 600 200 1/3 0.62 SSD 1000 600 30000 50 1.67 RAM 440 32 800,000 25,000 13.75 ��� GB IOPS�� �� SATA (0,0.31] TFS SAS (0.31,0.9] TDB SSD (0.9,1664] TSSD DRAM (1664è∞] TMEM Ws¢kf •  n/¢kfå •  D�:E�Q���Kˆ#n/ê –  Posix –  Mysql –  Memcachedèredis CFS – �ext3�Š� TFS vfs�Linux IO � � � �� CFS � mount#–t#cfs##data1/# read3/data1/hello3word.txt � b A >� � R >� vfs.read(/data1/hello3 world.txt) � cfs.read(/data1/hello3 world.txt) � TCPp m� Zs.read(/data1/hello3 world.txt) � TFS � CMEM – �Memcached�Š�  NoSQL E#4� �Memcached�Redis…�� TMEM� TSSD � TDB� ID� w9� kn�� CDB – �Mysql�Š� NoSQL msyql32 … mysql1 B 6 B 6 B 6 B 6 Ig¬ TGWÞ¸ Mysql Þ¸ yÊ>�±´ msyql32 … mysql1 msyql32 … mysql1 msyql32 … mysql1 TFS Master Slave N:1>� TSSD Á"xa³ NoSQL SQL POSIX E#4� 8b4� TMEM TSSD TDB /�4� TFS CMEM� CDB� CFS �