Types
DecoderErrorMode = enum demFatal, demReplacement
TextDecoderContext = object td*: TextDecoder n*: int oq*: seq[uint8] failed*: bool errorMode*: DecoderErrorMode
UnsafeSlice = object p*: ptr UncheckedArray[char] len*: int
Procs
proc `$`(sl: UnsafeSlice): string {....raises: [], tags: [], forbids: [].}
proc `&=`(s: var string; sl: UnsafeSlice) {....raises: [], tags: [], forbids: [].}
-
Append the slice sl to s without unnecessary copying.
Note that setLen is called on the string, which may zero out the target space before the copy.
proc decodeAll(iq: openArray[char]; charset: Charset): string {....raises: [], tags: [], forbids: [].}
- See above.
proc decodeAll(iq: openArray[char]; charset: Charset; success: var bool): string {. ...raises: [], tags: [], forbids: [].}
proc decodeAll(iq: openArray[uint8]; charset: Charset): string {....raises: [], tags: [], forbids: [].}
- Use td to decode iq, representing a complete contiguous input queue. Decoding errors are represented by U+FFFD replacement characters in the output.
proc decodeAll(iq: openArray[uint8]; charset: Charset; success: var bool): string {. ...raises: [], tags: [], forbids: [].}
- Decode iq using charset, with iq representing a complete contiguous input queue. When a decoding error occurs, success is set to false and an empty string is returned; otherwise, it is set to true and the decoder's full output is returned.
proc high(sl: UnsafeSlice): int {....raises: [], tags: [], forbids: [].}
proc initTextDecoder(charset: Charset): TextDecoder {....raises: [], tags: [], forbids: [].}
- Create a new TextDecoder instance from the charset. charset may be any value except csUnknown.
proc initTextDecoderContext(charset: Charset; errorMode = demReplacement; bufLen = 4096): TextDecoderContext {....raises: [], tags: [], forbids: [].}
-
Initialize a new text decoder context.
charset is the charset to decode the buffer with.
errorMode affects how errors are handled. With demReplacement, a U+FFFD replacement character is output when an error is encountered. With demFatal, decoding is aborted and the failed member of the TextDecoderContext is set to true.
bufLen is the size of the internal buffer in bytes.
proc toValidUTF8(iq: openArray[char]): string {....raises: [], tags: [], forbids: [].}
- Validate the UTF-8 string iq, replacing invalid characters with U+FFFD replacement characters.
proc validateUTF8Surr(s: openArray[char]): int {....raises: [], tags: [], forbids: [].}
- Analogous to std/unicode's validateUtf8, but also reports surrogates.
Iterators
iterator decode(ctx: var TextDecoderContext; iq: openArray[uint8]; finish: bool): UnsafeSlice {. ...raises: [], tags: [], forbids: [].}
-
Decodes the bytes provided in iq (input queue).
Streaming consumers should set finish to true when decoding the last chunk. (If you don't know which chunk will be the last, just pass an empty chunk after the stream is broken.)
Returns an UnsafeSlice object, which can be further processed as an openArray or a string. WARNING: this is simply a pointer into the input data and/or the output buffer. Never use an UnsafeSlice object after the iteration you received it in.
Templates
template toOpenArray(sl: UnsafeSlice): openArray[char] {..}
template toOpenArray(sl: UnsafeSlice; lo, hi: int): openArray[char] {..}
template toOpenArrayByte(sl: UnsafeSlice): openArray[uint8] {..}
template toOpenArrayByte(sl: UnsafeSlice; lo, hi: int): openArray[uint8] {..}