Jump to content

Implementation of emojis

Edit links
From Wikipedia, the free encyclopedia

The implementation of emojis on different platforms took place across a three-decade period, starting in the 1990s. Today, the exact appearance of emoji is not prescribed but can vary between fonts and platforms, much like different typefaces.

For example, the Apple Color Emoji typeface is proprietary to Apple, and can only be used on Apple devices (without additional hacking).[1] Different computing companies have developed their own fonts to display emoji, some of which have been open-sourced to permit their reuse.[2][3] Both color and monochrome emoji typefaces exist, as well as at least one animated design.[4]

Technical aspects

[edit]

JIS, Shift JIS and Private Use Area encodings

[edit]

Various, often incompatible, character encoding schemes were developed by the different mobile providers in Japan for their own emoji sets. When transmitted in Shift JIS on NTT DoCoMo, emoji symbols are specified as a two-byte sequence in the range F89F through F9FC (as expressed in hexadecimal). Emoji pictograms on au by KDDI are specified using the <img> tag,[citation needed] encoded in Shift JIS between F340 and F7FC,[5][6] or encoded in extended JIS X 0208 between 7521 and 7B73.[6] SoftBank Mobile emoji support colors and animation, and use different formats on 2G versus 3G:[7] in the 2G format, they are encoded in sequences using the Escape and Shift In control characters, whereas in the 3G format, they are encoded in Shift JIS between F741 and FBDE.[5][6] The SoftBank 3G format collides with the overlapping Shift JIS ranges used by the other vendors: for example, the Shift JIS representation F797 is used for a convenience store (🏪) by SoftBank, but for a wristwatch (⌚️) by KDDI.[5][6]

DoCoMo[6] and SoftBank[8] also developed their own schemes for representing their emoji sets in extended JIS X 0208 between 7522 and 7E38. These often matched the encodings of similar KDDI emoji where they existed: for example, the camera (📷) was represented in Shift JIS as F8E2 by DoCoMo, F6EE by KDDI, and F948 by SoftBank, but as 7670 in JIS by all three.[6][8]

All three vendors and Google (for Gmail) each developed at least one scheme for encoding their emoji in the Unicode Private Use Area (with au developing two);[7] DoCoMo, for example, used the range U+E63E through U+E757.[6] Mostly, these five schemes do not overlap, but au's primary private use scheme partly collides with SoftBank's.[7] Versions of iOS prior to 5.1 encoded emoji in the SoftBank private use area scheme, with later versions using standard Unicode.[9][10]

Supplementary Multilingual Plane support

[edit]

Most, but not all, emoji are included in the Supplementary Multilingual Plane (SMP) of Unicode. The SMP also includes, for example, ancient scripts such as Cuneiform or Egyptian hieroglyphs, some modern scripts such as Adlam or Osage, and special-use characters such as Musical Symbols or Mathematical Alphanumeric Symbols.[11]

Unicode was originally designed as a 16-bit encoding, which could be represented in a pure 16-bit form known as UCS-2. This corresponds to the Basic Multilingual Plane (BMP) of the Universal Coded Character Set. In Unicode 2.0, this was expanded to 17 planes (numbered 0 through 16, where the BMP is plane 0), and the first non-BMP characters were allocated in Unicode 3.1.[12] UCS-2 is now obsolete and deprecated in favour of UTF-16, a variable-width encoding which follows UCS-2 for the BMP, but extends it with four-byte codes representing non-BMP characters. Non-BMP characters (in the SMP and in other supplementary planes, such as additional hanzi in the Supplementary Ideographic Plane, including some of the Cantonese characters from HKSCS) now number in the tens of thousands.[12]

Some systems introduced prior to the advent of Unicode emoji were only designed to support characters in the BMP, on the assumption that non-BMP characters would rarely be encountered,[13] although failure to properly handle characters outside of the BMP precludes Unicode compliance.[12] For example, earlier versions of MySQL supported UCS-2 and a variant of UTF-8 excluding four-byte codes, thus not handling non-BMP characters correctly. Support for UTF-32 and full support for UTF-16 and UTF-8 (under the name utf8mb4) was added in version 5.5,[14] with utf8 retained as an alias for the up-to-three-byte version, although this is intended to be changed in the future.[15]

The introduction of Unicode emoji created an incentive for vendors to improve their support for non-BMP characters.[13] The Unicode Consortium notes that "[b]ecause of the demand for emoji, many implementations have upgraded their Unicode support substantially."[16]

Font format support

[edit]

Any operating system that supports adding additional fonts to the system can add an emoji-supporting font. However, inclusion of colorful emoji in existing font formats requires dedicated support for color glyphs. Not all operating systems have support for color fonts, so in these cases emoji might have to be rendered as black-and-white line art or not at all. There are four different formats used for multi-color glyphs in an SFNT font.[17][18] OpenType version 1.8 standardizes all four.

  • CBDT[18] is a multi-color raster format, using raw bitmap data or embedded PNG data.[19] It requires an auxiliary CBLC table to hold details about the image format,[19] and as such the format is sometimes called "CBDT/CBLC".[17]
  • COLR[18] is a multi-color vector format, using multiple single-color OpenType (TrueType or CFF) glyphs. A number of glyphs are assigned individual block colors and layered, and associated with a specified single-color base glyph. The layered glyphs are shown instead of the base glyph when the character is displayed as multi-color.[20] Colors are referenced from a palette, which is defined in a separate CPAL table,[20][21] and as such the format is sometimes called "COLR/CPAL".[17]
  • sbix is a multi-color raster format, using embedded JPEG, PNG or TIFF images. Including multi-color raster or vector glyphs as embedded PDF files is also permitted by the Apple Advanced Typography specification, but not by the OpenType specification, and is planned for future releases of iOS and macOS.[22][23]
  • SVG-in-OpenType (or OpenType-SVG) is a multi-color vector format (with support for embedded rasters),[18] using embedded SVG images. It requires the inclusion of a single-color TrueType or CFF glyph as well, serving as a fallback.[24] Being based on SVG, it includes support for color gradients.[24] An OpenType-SVG font may use a CPAL palette, similarly to COLR; however, this is not required.[21]

The COLR format was introduced by Microsoft, with Windows 8.1.[17] The CBDT format was introduced by Google and is supported on Android, while the competing sbix format was introduced by Apple, and is supported on macOS and iOS. SVG-in-OpenType was designed by Mozilla and Adobe as an industry standard.[18]

Some support for SVG-in-OpenType support has been added to newer updates of Windows 10, and to newer versions of iOS and macOS.[18] DirectWrite has supported all four since Windows 10 Anniversary Update; however, Windows only supports a subset of SVG-in-OpenType.[17] On the web, SVG-in-OpenType is supported by recent versions of Firefox, Safari and Microsoft Edge, but not by Google Chrome; Edge and Safari additionally support sbix, while Edge and Chrome support CBDT and all four support COLR.[18]

This means that color fonts may need to be supplied in several formats to be usable on multiple operating systems, or in multiple applications.

Internationalized domain names

[edit]
Main articles: Emoji domain and Internationalized domain name

A limited number of top-level domains allow registration of domain names containing emoji characters. Emoji-containing subdomains are also possible under any top-level domain.

Implementation by different platforms

[edit]

Google (Android and ChromeOS)

[edit]

Google's Noto fonts project includes the Noto Color Emoji font, which supplies color glyphs for emoji characters.[25] ChromeOS, through its inclusion of the Noto fonts, supports the emoji set introduced through Unicode 6.2. As of ChromeOS 41, Noto Color Emoji is the default font for most emoji.

Android devices support emoji differently depending on the operating system version. Google added native emoji support to Android in July 2013 with Android 4.3,[26] and to the Google Keyboard in November 2013 for devices running Android 4.4 and later.[27] Android 7.0 Nougat added Unicode 9 emoji, skin tone modifiers, and a redesign of many existing emoji.[28]

Emoji are also supported by the Google Hangouts application (independent of the keyboard in use), in both Hangouts and SMS modes.[29] Several third-party messaging and keyboard applications (such as IQQI Keyboard) for Android devices[30] provide plugins that allow the use of emoji. With Android 8 (Oreo), Google added a compatibility library that, if included by app developers, makes the latest Noto emoji available on any platform since Android 4.3.[31]

Stock Android systems include the Noto glyphs for emoji characters, although individual social media apps may use their own glyphs instead.[32] However, mobile phone vendors HTC and LG deployed variants of NotoColorEmoji.ttf with custom glyphs prior to 2017,[33] and Samsung still does.[34] Some Japanese mobile carriers used to equip branded Android devices with emoji glyphs that were closer to the original ones, but apparently have stopped updating these circa 2015.[clarification needed]

Apple

[edit]

Apple first introduced emoji to their desktop operating system with the release of OS X 10.7 Lion, in 2011. Users can view emoji characters sent through email and messaging applications, which are commonly shared by mobile users, as well as any other application. Users can create emoji symbols using the "Characters" special input panel from almost any native application by selecting the "Edit" menu and pulling down to "Special Characters", or by the key combination ⌘ Command+⌥ Option+T. Users can also create these symbols by switching the keyboard to Unicode, holding ⌥ Option and typing the Unicode hex input. For example, holding down ⌥ Option+2+6+3+A would create ☺. The desktop OS uses the Apple Color Emoji font that was introduced earlier in iOS. This provides users with full color pictographs.[35]

The emoji keyboard was first available in Japan with the release of iPhone OS version 2.2 in 2008.[36] The emoji keyboard was not officially made available outside of Japan until iOS version 5.0.[37] From iPhone OS 2.2 through to iOS 4.3.5 (2011), those outside Japan could access the keyboard but had to use a third party app to enable it. The first of such apps was developed by Josh Gare; emoji beginning to be embraced by popular culture outside Japan has been attributed to these apps.[38][39] iOS was updated to support Fitzpatrick skin-tone modifiers with version 8.3.[40]

OS X 10.9 Mavericks introduced a dedicated emoji input palette in most text input boxes within the Mac's existing Character Viewer using the key combination ⌘ Command+Ctrl+Space.[41] Optionally, the Fn key alone can be specified by the user in the keyboard preferences menu to bring up the Character Viewer. Since macOS Big Sur, the key is also labeled as 🌐 (globe) for consistency across macOS and iOS, which uses the globe key as a function key to switch to the emoji and other chosen international keyboard layouts.

Apple has revealed that the "face with tears of joy" is the most popular emoji among English speaking Americans. On second place is the "heart" emoji followed by the "Loudly Crying Face".[42][43]

On July 17, 2018, for the World Emoji Day, Apple announced that it will be adding 70 more emoji in its 2018 iOS update, including the long-awaited, red hair, white hair, curly hair and bald emoji.[44][45]

On September 12, 2017, Apple announced that the Messages app on the iPhones with Face ID would get "Animoji", which are versions of standard emoji that are custom-animated with the use of facial motion capture to reflect the sender's expressions. These Animoji can also utilize lip sync to appear to speak audio messages recorded by the sender. Apple had created 3D models of all standard emoji prior to its late-2016 OS updates from which the static default 2D graphics had been rendered. A select set of these models are being reused for creating still images and short animations dynamically.

With the release of iOS 12, Apple introduced "Memoji" that allows the use of an avatar that a user can use to personalize messages; this feature does not require Face ID.[46]

Mozilla (Firefox and Firefox OS)

[edit]

As part of the now-discontinued Firefox OS project, Mozilla developed an emoji font named FxEmojis.[47][48]

Mozilla also packages a version of Twitter's Twemoji font converted to a COLR/CPAL layered format font, named "Twemoji Mozilla".[49] Older versions[which?] of the latter Mozilla project instead packaged the EmojiOne font, as "EmojiOne Mozilla".[50] Since Firefox 50, emojis are rendered by the browser when the underlying platform lacks native support.[51]

Linux

[edit]

Ubuntu 18.04 and Fedora 28 support color emoji by default, using Noto Color Emoji.[52][53] Some Linux distributions require the installation of extra fonts.[54] Color emoji are supported by FreeType and Cairo.[55]

Microsoft Windows

[edit]

An update for Windows 7 and Windows Server 2008 R2 brought a subset of the monochrome Unicode set to those operating systems as part of the Segoe UI Symbol font.[56] As of Windows 8.1 Preview, the Segoe UI Emoji font is included, which supplies full-color pictographs. The plain Segoe UI font lacks emoji characters, whereas Segoe UI Symbol and Segoe UI Emoji include them.

Emoji characters are accessed through the onscreen keyboard's 😀 key, or through the physical keyboard shortcut ⊞ Win+..

Differently from macOS and iOS, color glyphs are only supplied when the application supports Microsoft's DirectWrite API, and Segoe UI Emoji is explicitly declared, otherwise monochrome glyphs appear.[57] Microsoft's COLR/CPAL format for multi-color fonts such as Segoe UI Emoji is supported by the current versions of several web browsers on Windows (including Firefox, Google Chrome, Internet Explorer and Microsoft Edge), but not by many graphics applications.[18]

Windows 10 Anniversary Update added Unicode 9 emoji.[58]

In August 2022, Microsoft open sourced more than 1,500 of its 3D emoji to let creators remix and customize them. The library is available on Figma and GitHub.[59]

Social media platforms

[edit]

Facebook and Twitter replace all Unicode emoji used on their websites with their own custom graphics.

Prior to October 2017, Facebook had different sets for the main site and for its Messenger service, where only the former provides complete coverage. Messenger now uses Apple emoji on iOS, and the main Facebook set elsewhere.[60] Facebook reactions are only partially compatible with standard emoji.[citation needed]

Twitter has released Twemoji, which is their emoji graphics together with a JavaScript library to handle them, under the Creative Commons CC-BY 4.0 license and the MIT open-source license, respectively.[61] Despite this, the Android and iOS Twitter apps use the emoji graphics that are native to the platform they are running on (Apple and Google), instead of the Twemoji graphics.

Other emoji font vendors

[edit]
EmojiOne 2.2 logo on the ticket emoji

EmojiOne version 2.2, an open-source font available under a free content license, supports the full emoji set in color through Unicode Emoji 3.0, i.e. Unicode 9.0. Newer versions of EmojiOne, since renamed JoyPixels,[62] support more recent Unicode Emoji versions, and use a stricter license that disallows the redistribution of vector images, while version 2.x is "no longer supported or distributed".[63] EmojiTwo, an open-source fork of EmojiOne 2.2, aims to add all emoji from 2017 and later.

The font Symbola contains all emoji through version 10.0 as normal monochrome glyphs. Through version 10, Symbola was made available without a license nor any restrictions on use; beginning with version 11 in 2018, Symbola has been copyrighted with a ban on commercial use and derivative works. Other typefaces including a significant number of emoji characters include Noto Emoji, Adobe Source Emoji, and Quivira.

Footnotes

[edit]
  1. ^ Notes on the format of the gmojiraw.txt data file from Google/AOSP: file is mostly tab-separated, except that columns for a given vendor after a non-empty substitute string column are skipped altogether on a per-line basis, so the columns of two given lines do not necessarily line up. First four columns give the Google private use code points (in hexadecimal), the UTF-16 and UTF-8 thereof, and a Google-assigned name, which are followed by columns for au, then columns for DoCoMo, then columns for SoftBank. Columns for a given vendor consist of a substitute string followed by (IFF the substitute string is empty) a decimal ordinal, a hexadecimal Shift JIS code in the region beyond JIS X 0208, a hexadecimal private-use Unicode code point, a hexadecimal 7-bit JIS code and (for au only) an alternative Shift JIS code corresponding to the 7-bit JIS code. Characters which exist in a given vendor's Shift JIS scheme but not its 7-bit JIS scheme have the 7-bit code for that vendor listed as 222E (i.e. the geta mark in JIS X 0208). Some vendor mappings are approximations or to sequences (delimited within fields with +).

References

[edit]
  1. ^ "[MOD] Apple Color Emoji system-wide for KitKat+ (updated with unicorns)". XDA Developers. 10 December 2013. Retrieved January 15, 2015.
  2. ^ Davidson, Mike. "Open sourcing Twitter emoji for everyone". Twitter developer blog. Twitter. Retrieved January 15, 2015.
  3. ^ "Emoji One: Open Source Emoji". Emoji One. Archived from the original on August 27, 2017. Retrieved January 15, 2015.
  4. ^ El Khoury, Rita (December 11, 2014). "Woohoo! Animated Emoji Easter Eggs Overload The Latest Hangouts With Their Cuteness, Hehehehe". Android Police. Retrieved January 15, 2015.
  5. ^ a b c Unicode Consortium. "Emoji Sources". Unicode Character Database.
  6. ^ a b c d e f g Scherer, Markus; Davis, Mark; Momoi, Kat; Tong, Darick; Kida, Yasuo; Edberg, Peter. "Emoji Symbols: Background Data—Background data for Proposal for Encoding Emoji Symbols" (PDF). UTC L2/10-132.
  7. ^ a b c Kawasaki, Yusuke (2010). Emoji encodings and cross-mapping tables in pure Perl.
  8. ^ a b Android Open Source Project (2009). "GMoji Raw". Skia Emoji.[a]
  9. ^ "Apple iOS 5.1". Emojipedia.
  10. ^ "Apple iPhone OS 2.2". Emojipedia.
  11. ^ Everson, Michael; McGowan, Rick; Whistler, Ken; Umamaheswaran, V.S. (2020-07-22). "Roadmap to the SMP". Revision 13.0.3.
  12. ^ a b c Lunde, Ken (2009). CJKV Information Processing (2nd ed.). Sebastopol CA.: O'Reilly Media. p. 200. ISBN 978-0-596-51447-1.
  13. ^ a b Chupov, Sergey (2019-06-06). "How We Store Emojis in Your Database, or Why We Got Rid of the Extended String Data Type". Backendless Corporation.
  14. ^ Bushuev, Leonid. "4-bytes UTF-8 characters cause "Incorrect string value" error in MySQL". TeamCity YouTrack.
  15. ^ "10.10.1 Unicode Character Sets". MySQL 8.0 Documentation. Archived from the original on 2020-08-10.
  16. ^ "Don't emoji detract from the other work of the consortium?". Frequently Asked Questions: Emoji and Pictographs. Unicode Consortium.
  17. ^ a b c d e Microsoft (2018-05-31). "Color Fonts". Microsoft Docs.
  18. ^ a b c d e f g h "What's inside color fonts?". Color Fonts - Get ready for the revolution!.
  19. ^ a b Microsoft (31 May 2024). "CBDT — Color Bitmap Data Table". OpenType spec.
  20. ^ a b Microsoft (31 May 2024). "COLR — Color Table". OpenType spec.
  21. ^ a b Microsoft (29 May 2024). "CPAL — Color Palette Table". OpenType spec.
  22. ^ Microsoft (30 May 2024). "sbix — Standard Bitmap Graphics Table". OpenType spec.
  23. ^ Apple. "The 'sbix' table". TrueType Reference Manual.
  24. ^ a b Microsoft (30 May 2024). "SVG — The SVG (Scalable Vector Graphics) table". OpenType spec.
  25. ^ "Noto Color Emoji". Google Noto Fonts.
  26. ^ Cabebe, Jaymar. "Google Android 4.3 is here, and it tastes like Jelly Bean". CNET.
  27. ^ "Google adds SMS to Hangouts Android app, Emoji to KitKat keyboard". November 7, 2013. Retrieved April 17, 2014.
  28. ^ "Android 7.0 Nougat Emoji Changelog". August 22, 2016. Retrieved August 23, 2016.
  29. ^ "Hangouts – Google Play". Retrieved April 17, 2014.
  30. ^ "emoji – Google Play". Market.android.com. Retrieved November 9, 2012.
  31. ^ Ion, Florence (July 24, 2017). "Fewer Empty Boxes for Android Users". Emojipedia.
  32. ^ Emojipedia. "Google Emoji List".
  33. ^ Emojipedia. "LG Emoji List".
  34. ^ Emojipedia. "Samsung Emoji List".
  35. ^ "Access and Use Emoji in Mac OS X". Osxdaily.com. August 20, 2011. Retrieved January 18, 2014.
  36. ^ "Apple releases iPhone Software v2.2". AppleInsider. Archived from the original on March 1, 2017. Retrieved February 28, 2017.
  37. ^ "Standard Emoji keyboard arrives to iOS 5, here's how to enable it". 9to5Mac. June 8, 2011. Retrieved February 28, 2017.
  38. ^ "Young App Creators Earning Thousands A Day". Sky News. Retrieved February 28, 2017.
  39. ^ "The man who brought us the Emoji". O2. October 16, 2015. Archived from the original on September 6, 2018. Retrieved February 28, 2017.
  40. ^ Underhill, Allison (April 10, 2015). "The 'Diversity' of Emojis". The Huffington Post. Retrieved December 15, 2015.
  41. ^ Cipriani, Jason (October 23, 2013). "How to access emoji in OS X 10.9 Mavericks". CNET. Retrieved January 18, 2014.
  42. ^ "Apple Says 'Face With Tears of Joy' is Most Popular Emoji in United States Among English Speakers". Retrieved November 3, 2017.
  43. ^ "😃 Emoji People and Smileys Meanings". emojipedia.org. Retrieved November 3, 2017.
  44. ^ Kelly, Heather. "Redheads, lobsters and cupcakes: Apple shows off new iOS emojis". CNNMoney. Retrieved 2018-07-17.
  45. ^ "Apple emoji will soon include people with curly hair, white hair and superpowers". www.msn.com. Retrieved 2018-07-17.
  46. ^ "How to Create and Use Memoji and Animoji on an iPhone". How to Geek. 2020-01-18. Retrieved 2020-03-20.
  47. ^ Mozilla (June 15, 2021). "FxEmojis — a friendly emoji set from Mozilla". GitHub.
  48. ^ Emojipedia. "Mozilla Emoji List — Emojis for Firefox OS". Emojipedia.
  49. ^ Mozilla (July 8, 2021). "twemoji-colr: Twemoji font in COLR/CPAL layered format". GitHub.
  50. ^ Mozilla. "emojione-colr: Project to create a COLR/CPAL-based color OpenType font from the EmojiOne collection of emoji images". GitHub. v0.2.2.
  51. ^ "Firefox 50.0, See All New Features, Updates and Fixes".
  52. ^ "Ubuntu 18.04 Will Support Color Emoji – OMG! Ubuntu!". OMG! Ubuntu!. 2017-11-08. Retrieved 2018-06-09.
  53. ^ "What's New in Fedora 28 Workstation – Fedora Magazine". Fedora Magazine. 2018-05-01. Retrieved 2018-06-09.
  54. ^ Petherbridge, Noah (April 4, 2013). "Make Emoji Work in Linux". Kistle blog. Retrieved October 7, 2014.
  55. ^ LEMBERG, Werner. "[ft-announce] FreeType now supports color emojis". Retrieved April 13, 2018.
  56. ^ "An update for the Segoe UI symbol font in Windows 7 and in Windows Server 2008 R2 is available". Microsoft Support.
  57. ^ "Script and Font Support in Windows". Microsoft. Archived from the original on September 23, 2017. Retrieved August 23, 2017.
  58. ^ "Windows 10 Anniversary Update Adds Over 52,000 New Emojis, Including NinjaCat". Windows Central. August 2, 2016. Retrieved August 23, 2017.
  59. ^ Warren, Tom (10 August 2022). "Microsoft open sources its 3D emoji to let creators remix and customize them". The Verge. Retrieved 14 August 2022.
  60. ^ Burge, Jeremy (2 October 2017). "Facebook Discontinues Messenger Emojis". Emojipedia.
  61. ^ "GitHub – twitter/twemoji: Twitter Emoji for Everyone". GitHub. July 20, 2017. Retrieved September 24, 2017.
  62. ^ Neufeld, Sarah (2019). "EmojiOne is Now JoyPixels".
  63. ^ JoyPixels (July 8, 2021). "emoji-toolkit". GitHub.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Implementation_of_emojis&oldid=1234729940"