Support for True Color (16 millions colors) in Terminal app

Assigned To
Samis2001, billiob, berkeleynerd and 4 others
  • Restricted Project
"Like" token, awarded by berkeleynerd.

Here's a test case

printf "\x1b[38;2;255;100;0mTRUECOLOR\x1b[0m\n"



According to Wikipedia[1], this is only supported by xterm and konsole.

It's a common confusion about terminal colors... Actually we have this:

  • plain ascii
  • ansi escape codes (16 color codes with bold/italic and background)
  • 256 color palette (216 colors+16gray + ansi) (colors are 24bit)
  • 24bit true color (8*8*8 colors (aka 16 milion)

The 256 color palete is configured at start, and it's a 6*6*6 cube of
colors, each of them defined as a 24bit (8*8*8 rgb) color.

This means that current support can only display 256 *different* colors
in the terminal, while truecolor means that you can display 16 milion
different colors at the same time.

Truecolor escape codes doesnt uses a color palete. It just specifies the
color itself.


Here is another terminals discussions:

Now supporting truecolor:

Parsing ANSI color sequences, but approximating them to 256 palette:

Not supporting truecolor:

Here are discussions/bugs for common console applications:

akochkov created this task.Via WebJan 10 2014, 9:59 AM
akochkov edited the task description. (Show Details)
akochkov added a subscriber: akochkov.
billiob triaged this task as "Pending on user input" priority.Via WebJan 10 2014, 10:04 AM
billiob added a project: Restricted Project.
egmont added a subscriber: egmont.Via WebJan 11 2014, 2:15 PM
akochkov edited the task description. (Show Details)Via WebJan 19 2014, 6:06 PM
akochkov edited the task description. (Show Details)Via WebJan 19 2014, 6:09 PM
egmont edited the task description. (Show Details)Via WebJan 19 2014, 6:15 PM
raster closed this task as "Wontfix".Via WebJan 23 2014, 12:21 AM
raster claimed this task.
raster added a subscriber: vtorri.

when i started terminology and before that when working on textgrid with @vtorri, i made the decision already to not support this. i knew about it. this is a conscious decision to not support it BECAUSE:

  1. it means breaking ABI of evas (or having to extend the api with some textgrid2/rgb api with rgb capable structs)
  2. increasing memory footprint per char in textgrid (we already use 8 bytes - we now basically go up to 16 bytes due to alignment/padding)
  3. increasing scrollback (going to 16bytes from 8) doubling out scrollback memory footprint

all for a SLIGHT improvement in defining the bg and fg color of a coarse grid of text chars (your spatial resolution is highly limited thus the value of fine-grained color choices is low and the cost is high).

i gave this real thought and it's designed simply to not be supported due to the above. if that is a deal breaker for users, then so be it. :(

egmont added a comment.Via WebJan 23 2014, 8:57 AM

Please allow me to put my 2 cents in this.

The reporter of this bug and some other guys (myself included) have recently put some effort in getting true color terminal support more widespread. At this moment it's hard to tell if it's going to be successful. Unfortunately both ncurses and slang would require API extension or ABI incompatible change, just like textgrid for Terminology. It's unclear whether their developers are willing to go this direction any time soon. On the other hand, probably the two leading terminal apps, vim and emacs use their own method for handling the screen, there's a patch for true color support for both of them, and hopefully those will be accepted by mainstream.

As for the memory concerns: Let's talk about 12 bytes instead of 16, using attribute(packed). I'm quite sure that the cost of reading from non-aligned offset on 64-bit architectures is negligible compared to the cost of what's going to happen to that data (glyph lookup, drawing on the screen). I have a feeling that a scrollback of 10.000 lines (the maximum allowed by Terminology, compared to the default 2.000), 100 columns per terminal, 25 terminals is probably a heavier-than-average usage. Here true colors would require an extra 100 MB. It's not negligible, but it doesn't sound the end of the world either. If this 50% increase in memory usage is an issue, I'd like to mention that probably the current way of storing the scrollback is already way too wasteful. Vte (gnome-terminal), for example, uses UTF-8 for the scrollback (but not the onscreen part), with runlength encoding of identical attributes. Meaning that storing a character typically takes somewhere between hardly more than 1 byte to maybe around 4 bytes in average, including true color support. I'm just mentioning it as an idea to consider.

You mentioned true color would give only a SLIGHT improvement. For most users this is probably true, but for some, true colors would give a significant usability boost. I myself have hit the serious limitations of 256 colors when designing a skin for midnight commander. I imagined a certain color, but the 256 color palette contained nothing even close to that. In my opinion, not having access to any color is just LAME. YMMV.

Terminology's TODO suggests it wants to become a "first-class terminal". It has all the bells and whistles in the chrome one could imagine that are not strongly related to the terminal itself, e.g. image preview, background video etc. But for some reason, developers seem to be resistant in making the real core, the actual terminal emulation first-class. (See also T56 and T631: no support for bold and italic). As such, I really don't see a consistent vision in what Terminology wants to be.

Anyway, your project, your call.

raster added a comment.Via WebJan 23 2014, 4:59 PM
  1. never am i going to use attribute(packed) - why? the struct for the text glyph is in a public header. so it's 16. :) (not to mention poor poor poor ARM or anything non-x86 if we pack stuff). there is a real cost. in fact it means we are broken/screwed on ARM/non-x86 as we return a pointer to an array of text cell structs for a row for speed reasons - so packing is not going to help no matter what as architectures other than x86 literally cause faults (in some cases SIBGUS or SIGSEGV, in others kernels have fixup code that drags you into the realm of continental drift).
  2. yup - others have the same issue abi-wise. :(
  3. terminology allows much more than 10,000 lines - just the slider in the gui stops there. :) the limit in code is about 2 billion lines... :)
  4. an extra 100mb is not negligible. 100mb is stuff that keeps me awake at night. our entire desktop environment including base os boot, kernel, xserver, wallpapers etc. boots on 64mb with no swap and compositing on... and can run apps. 100mb EXTRA is the stuff of nightmares. this was the point i was trying to make. it was a well considered decision. :(
  5. terminology stores scrollback per line as an array from line start until the last non-empty char (so short lines consume less). each char is a cell struct. we USE the complete int for the glyph not just for the char but also for inlined images (we use high bits and beyond-unicode glyph point ranges for these to mark the media item ID - 8192 of those at a time) and then up to a 512x152 char cell blob for the chars that make it up - every char has to be accounted for with a relative x,y position to refcount the media item and handle overwrite). so it's actually used, so utf-8 is not viable. not only that but we compress scrollback - we use lz4. so yes - the 100mb extra above becomes 50mb or so after compression - maybe less, but it still costs and requires an abi break (never going to happen any time soon), OR a parallel "glyph2" api to switch in efl. and if i'm going RGB - i'm going RGBA not just RGB. so i would need to add 8 more bytes at the end of RGBA fg/bg (i'd consume 8 byts either way withotu the A and A can be used as the on/off switch for 24bit mode - if 0 then stay in old mode) i'm not breaking abi/api only to extend a limited amount ... only to then find people want to set alpha too. but - this is at the efl/evas level, not terminology, and evas textgrid is a grid of text, not a terminal emulator, so it needs to be more generic here. :) and it's not 1 byte.. you have to store fg/bg color index, other attributes (bold, faint, italic, double width flag, underline, blink, blink2, inverse, invisible, strikethrough, 256 color mode, intense fg/bg, then metadata generated while parsing terminal like autowrap flag, newline, tab...) yes - they need storage per char. terminology doesn't get to be as fast as it is without making specific decisions on what we store and how, and how we rebuild content from history. so *IF* we used 1 byte per char we'd need another 4 bytes just for attributes. we could do specialized encoding of each scrollback line and make it different to a live line where we pack data bit-wise, and use RLE techniques so runs of data with the same attributes etc. get more packed before we stuff through lz4, but now we're talking a whole new world of complexity beyond just punting our scrollback through lz4 as-is and getting it back out of compression in its expanded form. :)
  6. as for "skins" i think we have a different view/take on this. my view is that you don't skin mc or vim etc... that you configure your favorite color palette in your terminal (terminology allows this - including the entire 256 color palette too - most terminals allow at least customizing the basic pre-256 color set) to your liking and then you are using "enumerated colors" from a palette that match the kind of colors you want - eg what solarized does to mute colors. but as you said - for most users it's not that relevant, but the memory usage is... :)
  7. bold and italic are explicit choices. they both violate the text grid. they either force characters to exit their cell grid and thus either have to be clipped or overflow and thus basically be "wrong" (also then selections and much more begin to have trouble being right) OR you have to expand your cell grid to accommodate the largest such char that now is far wider/taller due to italic/bold attributes. to YOU maybe these things are important - to me just using a lighter color instead of bold is perfectly fine, and italic is something i have never even missed as no terminal i have used since 1994 actually had it working (i had to configure xterm or rxvt to use specific italic capable fonts and i never did - to make it happen).

so the reason i say no is that there is a real cost to doing 24bit terminal support. i chose not to in design due to the dearth of support for it in general in terminal apps, the dearth of seeing anyone actually use it - people don't even use 256 color support (the vast majority) so even that was a stretch, but i went there. seeing people even customize their shell is rare - everyone i see uses defaults. their prompts are basic "$" not a spot of color in most cases. the few i do see customized are using the old basic colors, not even 256. their vims are default as well. so until people even embrace 256 colors... why go all the way to 24bit? the inline media actually consumes no space in history (beyond path/uri) as terminology remembers the path/uri only - not the data. it's created/destroyed on the fly. 24bit will be a cost regardless, and that all still has the core problem - abi break in evas - and that is not going to happen (not until efl 2.0), or a parallel api.

remember this too. we have to prioritize. indeed i appreciate that you have a crusade. improve 24bit color support. but you have to see things from our angle too. look at this:


that's our current active bug list, and incoming patch review queue. they don't go down in size. if those were basically empty ... i might consider the work here, but they are far from it - they grow and grow. we have to prioritize.

so we spend time on a feature that has minimal benefit for most people, comes with a memory footprint hike and/or the need to write yet more exotic compression and storage techniques to combat the extra overhead, and an abi break or a parallel api extension to maintain... or do we say "no". and try and get what we have under control and get through feature lists we already have even before terminology existed - go through its existing TODO list (TODO file in git), let alone handle the bugs (remember we do much more than just a terminal... a whole toolkit, wm/compositor/fm/de etc. etc.) and at some point someone has to be the nasty evil guy who says no. unfortunately today that guy is me :(. i have to say no, because on the balance of the work needed, repercussions in api/abi, the fact that even ncurses and slang are also saying no, and the general usefulness vs what is already there, it just has to drop off the bottom of the list. i have just spent my ENTIRE week doing NOTHING except responding to patch reviews and bug filings. i have not even had time to catch up on my email and people are pestering me there to answer questions and discuss development. i'm not the only one swamped. and that is not even touching the stuff i actually should do for my job - stuff that matters to the people who pay me and put a roof over my head and food in my mouth... and this is the case for pretty much all us devs - we have to prioritize. often what is important to us/our employers ALSO is important to others- then everyone winds. sometimes we take some time out to do something for fun or to be nice, but we can't do this all the time.

if you are offering to DO the work then it's a different matter (we still have the maintenance and abi/extension api issue), but if you are asking US to stop other things we do and do this, then someone has to make a call somewhere. not all answers are yes. we say yes often, but sorry - this time - no. :( as i said - come back when our buglists are smaller - or help make them small. :)

i hope this doesn't discourage you too much. keep at it, and maybe - just maybe, at a future date this will come back around when abi/api can be changed and we might adapt to have support then we can add it to the terminal too. :)

egmont added a comment.Via WebJan 23 2014, 5:37 PM

Thanks for the long response. The fact that you took time to write such a long reply suggests that you probably got offended by my last comment, I'm terribly sorry if that's the case, please rest assured that by no means did I mean to hurt you or criticize the work you're doing, I'm sorry if it went wrong. I purely wanted to express my different opinion while accepting your decision. And indeed, you have many todo items that are more important than this one. Please keep up the good work, and thanks for creating Terminology at the first place :)

raster added a comment.Via WebJan 23 2014, 7:27 PM

oh god no! not offended! i just thought you had certain assumptions/views on what is actually going on inside efl/terminology that were not fully filled in. that's the point of a dialog. to fill them in. :) you did fill in info for me for example - that 24 bit support exists in xterm/mlterm but they approximate to 256 color. THIS is possibly doable as it just requires the parsing then approximation and we store/retain the approximation. :) the info that ncurses/slang are in the same abi problem boat leads me to know that there likely isn't a big rush/need for this until they solve their issues. :) i just want to take the time to let you know that i haven't come to this decision lightly. i have given it real thought, and to indicate how much i've thought about it i need to explain a few things (in detail) so you appreciate that this isn't a flippant decision. i take saying "no" seriously. i hate to say no to features, but sometimes i have to. it isn't personal. it isn't because i have my head stuck in the sand. it's a thought out decision - at least for now, to not break abi or add the api complexity and the memory overhead. at a later date when a lot of other stuff is more mature and under control - sure. i'm willing to re-evaluate things. we have a lot we want to add to efl - like pushing lz4 compression into efl as a standard api. that will remove it from terminology and mean rle compression is the next problem. i'd love to solve the rle thing in a more generic way - maybe have some way to describe bytes/bitfields to an rle compressor with some hints of 'this changes often, this does not' and see if it can rle (de)compress before/after going through lz4 which may nuke a lot of the overhead in scrollback - and this is an efl2.0 thing for the abi/api changes to textgrid, but until we have all these things in place in a mature and solid way... i don't want to do this 24bit support here. :)

berkeleynerd added a subscriber: berkeleynerd.Via WebAug 18 2015, 1:05 PM
Herald added a subscriber: billiob. · View Herald TranscriptVia HeraldAug 18 2015, 1:05 PM
berkeleynerd awarded a token.Via WebAug 18 2015, 1:05 PM
berkeleynerd added a comment.Via WebAug 18 2015, 1:09 PM

FWIW, I would like 24-bit color support for the same reason I sought out terminology...I'm trying to find the best console experience one can have in the year 2015. If there were a bounty for this feature I would certainly chip-in to try to make it happen. All points on both sides were valid. Looks like we need a hero to implement this feature.

billiob changed the visibility of this Maniphest Task from "All Users" to "Public (No Login Required)".Via WebAug 24 2015, 2:18 AM
billiob reopened this task as "Open".Via WebTue, Sep 8, 1:18 PM
billiob added a comment.Via WebTue, Sep 8, 1:27 PM

This change will need to have an up-to-date EFL.
The best way to help is to get EFL/elementary/terminology correctly packaged in distribution.
I'd like to be that hero but my TODO-list on terminology is already quite heavy and this feature is not in my top priorities :(

I wouldn't mind having to review more (good) patches!

Samis2001 added a subscriber: Samis2001.Via WebSun, Sep 13, 8:07 AM

I would appreciate this feature as well, unfortunately I'm not a C programmer so I can't help. Hope someone implements it as a patch.

billiob raised the priority of this task from "Pending on user input" to "Wishlist".Via WebSat, Sep 26, 3:11 PM

Add Comment