C++ String Class Assignment Operator Python

I am new to Cython but am very interested in the smooth interface between C/C++ and Python which it promises. I have hit an issue which I cannot work through on my own.

Executive summary

When I try to assign a stack-allocated C++ object wrapped in Cython with the value of another one, I get a segfault. I cannot understand why this is occurring.

Details

I am working with a code base with a class to read and write a custom format for a binary serialization of some data. Below is a minimal reimplementation of the C++ code reading from the binary. The class is the main interface into the binary data files. It uses a generator-like object, implemented as a subclass , to read through the files holding the binary data until the file is exhausted. Individual records are typed as an inner class, .

I'd like to expose this library (written in C++) to some folks who primarily use Python. It seems natural to make use of a generator construct in Python too, so that one could use the following interface in Python:

In my implementation of the iteritems method (see below), everything is working fine except when I try to assign a value to a stack-allocated variable of the generator type. Here are my Cython description and implementation files.

Using a dummy file, with contents "data line 1\ndata line2 (etc)", what I really see is this:

The problem line is , which I think means my problem comes from my wrapper on the but I cannot unpack the exact cause and find a solution. Any help would be greatly appreciated!

pythonc++cython

5. Built-in Types¶

The following sections describe the standard types that are built into the interpreter.

Note

Historically (until release 2.2), Python’s built-in types have differed from user-defined types because it was not possible to use the built-in types as the basis for object-oriented inheritance. This limitation no longer exists.

The principal built-in types are numerics, sequences, mappings, files, classes, instances and exceptions.

Some operations are supported by several object types; in particular, practically all objects can be compared, tested for truth value, and converted to a string (with the repr() function or the slightly different function). The latter function is implicitly used when an object is written by the function.

5.1. Truth Value Testing¶

Any object can be tested for truth value, for use in an or condition or as operand of the Boolean operations below. The following values are considered false:

  • zero of any numeric type, for example, , , , .

  • any empty sequence, for example, , , .

  • any empty mapping, for example, .

  • instances of user-defined classes, if the class defines a or method, when that method returns the integer zero or value . [1]

All other values are considered true — so objects of many types are always true.

Operations and built-in functions that have a Boolean result always return or for false and or for true, unless otherwise stated. (Important exception: the Boolean operations and always return one of their operands.)

5.2. Boolean Operations — , , ¶

These are the Boolean operations, ordered by ascending priority:

OperationResultNotes
if x is false, then y, else x(1)
if x is false, then x, else y(2)
if x is false, then , else (3)

Notes:

  1. This is a short-circuit operator, so it only evaluates the second argument if the first one is false.
  2. This is a short-circuit operator, so it only evaluates the second argument if the first one is true.
  3. has a lower priority than non-Boolean operators, so is interpreted as , and is a syntax error.

5.3. Comparisons¶

Comparison operations are supported by all objects. They all have the same priority (which is higher than that of the Boolean operations). Comparisons can be chained arbitrarily; for example, is equivalent to , except that y is evaluated only once (but in both cases z is not evaluated at all when is found to be false).

This table summarizes the comparison operations:

OperationMeaningNotes
strictly less than 
less than or equal 
strictly greater than 
greater than or equal 
equal 
not equal(1)
object identity 
negated object identity 

Notes:

  1. can also be written , but this is an obsolete usage kept for backwards compatibility only. New code should always use .

Objects of different types, except different numeric types and different string types, never compare equal; such objects are ordered consistently but arbitrarily (so that sorting a heterogeneous array yields a consistent result). Furthermore, some types (for example, file objects) support only a degenerate notion of comparison where any two objects of that type are unequal. Again, such objects are ordered arbitrarily but consistently. The , , and operators will raise a exception when any operand is a complex number.

Non-identical instances of a class normally compare as non-equal unless the class defines the method or the method.

Instances of a class cannot be ordered with respect to other instances of the same class, or other types of object, unless the class defines either enough of the rich comparison methods (, , , and ) or the method.

CPython implementation detail: Objects of different types except numbers are ordered by their type names; objects of the same types that don’t support proper comparison are ordered by their address.

Two more operations with the same syntactic priority, and , are supported only by sequence types (below).

5.4. Numeric Types — , , , ¶

There are four distinct numeric types: plain integers, long integers, floating point numbers, and complex numbers. In addition, Booleans are a subtype of plain integers. Plain integers (also just called integers) are implemented using in C, which gives them at least 32 bits of precision ( is always set to the maximum plain integer value for the current platform, the minimum value is ). Long integers have unlimited precision. Floating point numbers are usually implemented using in C; information about the precision and internal representation of floating point numbers for the machine on which your program is running is available in . Complex numbers have a real and imaginary part, which are each a floating point number. To extract these parts from a complex number z, use and . (The standard library includes additional numeric types, that hold rationals, and that hold floating-point numbers with user-definable precision.)

Numbers are created by numeric literals or as the result of built-in functions and operators. Unadorned integer literals (including binary, hex, and octal numbers) yield plain integers unless the value they denote is too large to be represented as a plain integer, in which case they yield a long integer. Integer literals with an or suffix yield long integers ( is preferred because looks too much like eleven!). Numeric literals containing a decimal point or an exponent sign yield floating point numbers. Appending or to a numeric literal yields an imaginary number (a complex number with a zero real part) which you can add to an integer or float to get a complex number with real and imaginary parts.

Python fully supports mixed arithmetic: when a binary arithmetic operator has operands of different numeric types, the operand with the “narrower” type is widened to that of the other, where plain integer is narrower than long integer is narrower than floating point is narrower than complex. Comparisons between numbers of mixed type use the same rule. [2] The constructors , , , and can be used to produce numbers of a specific type.

All built-in numeric types support the following operations. See The power operator and later sections for the operators’ priorities.

OperationResultNotes
sum of x and y 
difference of x and y 
product of x and y 
quotient of x and y(1)
(floored) quotient of x and y(4)(5)
remainder of (4)
x negated 
x unchanged 
absolute value or magnitude of x(3)
x converted to integer(2)
x converted to long integer(2)
x converted to floating point(6)
a complex number with real part re, imaginary part im. im defaults to zero. 
conjugate of the complex number c. (Identity on real numbers) 
the pair (3)(4)
x to the power y(3)(7)
x to the power y(7)

Notes:

  1. For (plain or long) integer division, the result is an integer. The result is always rounded towards minus infinity: 1/2 is 0, (-1)/2 is -1, 1/(-2) is -1, and (-1)/(-2) is 0. Note that the result is a long integer if either operand is a long integer, regardless of the numeric value.

  2. Conversion from floats using or truncates toward zero like the related function, . Use the function to round downward and to round upward.

  3. See Built-in Functions for a full description.

  4. Deprecated since version 2.3: The floor division operator, the modulo operator, and the function are no longer defined for complex numbers. Instead, convert to a floating point number using the function if appropriate.

  5. Also referred to as integer division. The resultant value is a whole integer, though the result’s type is not necessarily int.

  6. float also accepts the strings “nan” and “inf” with an optional prefix “+” or “-” for Not a Number (NaN) and positive or negative infinity.

  7. Python defines and to be , as is common for programming languages.

All types (, , and ) also include the following operations:

5.4.1. Bitwise Operations on Integer Types¶

Bitwise operations only make sense for integers. Negative numbers are treated as their 2’s complement value (this assumes a sufficiently large number of bits that no overflow occurs during the operation).

The priorities of the binary bitwise operations are all lower than the numeric operations and higher than the comparisons; the unary operation has the same priority as the other unary numeric operations ( and ).

This table lists the bitwise operations sorted in ascending priority:

OperationResultNotes
bitwise or of x and y 
bitwise exclusive or of x and y 
bitwise and of x and y 
x shifted left by n bits(1)(2)
x shifted right by n bits(1)(3)
the bits of x inverted 

Notes:

  1. Negative shift counts are illegal and cause a to be raised.
  2. A left shift by n bits is equivalent to multiplication by . A long integer is returned if the result exceeds the range of plain integers.
  3. A right shift by n bits is equivalent to division by .

5.4.2. Additional Methods on Integer Types¶

The integer types implement the abstract base class. In addition, they provide one more method:

()¶
()¶

Return the number of bits necessary to represent an integer in binary, excluding the sign and leading zeros:

More precisely, if is nonzero, then is the unique positive integer such that . Equivalently, when is small enough to have a correctly rounded logarithm, then . If is zero, then returns .

Equivalent to:

>>> n=-37>>> bin(n)'-0b100101'>>> n.bit_length()6
defbit_length(self):s=bin(self)# binary representation: bin(-37) --> '-0b100101's=s.lstrip('-0b')# remove leading zeros and minus signreturnlen(s)# len('100101') --> 6

5.4.3. Additional Methods on Float¶

The float type implements the abstract base class. float also has the following additional methods.

()¶

Return a pair of integers whose ratio is exactly equal to the original float and with a positive denominator. Raises on infinities and a on NaNs.

()¶

Return if the float instance is finite with integral value, and otherwise:

>>> (-2.0).is_integer()True>>> (3.2).is_integer()False

Two methods support conversion to and from hexadecimal strings. Since Python’s floats are stored internally as binary numbers, converting a float to or from a decimal string usually involves a small rounding error. In contrast, hexadecimal strings allow exact representation and specification of floating-point numbers. This can be useful when debugging, and in numerical work.

()¶

Return a representation of a floating-point number as a hexadecimal string. For finite floating-point numbers, this representation will always include a leading and a trailing and exponent.

(s

Class method to return the float represented by a hexadecimal string s. The string s may have leading and trailing whitespace.

Note that is an instance method, while is a class method.

A hexadecimal string takes the form:

where the optional may by either or , and are strings of hexadecimal digits, and is a decimal integer with an optional leading sign. Case is not significant, and there must be at least one hexadecimal digit in either the integer or the fraction. This syntax is similar to the syntax specified in section 6.4.4.2 of the C99 standard, and also to the syntax used in Java 1.5 onwards. In particular, the output of is usable as a hexadecimal floating-point literal in C or Java code, and hexadecimal strings produced by C’s format character or Java’s are accepted by .

Note that the exponent is written in decimal rather than hexadecimal, and that it gives the power of 2 by which to multiply the coefficient. For example, the hexadecimal string represents the floating-point number , or :

Applying the reverse conversion to gives a different hexadecimal string representing the same number:

[sign]['0x']integer['.'fraction]['p'exponent]
>>> float.fromhex('0x3.a7p10')3740.0
>>> float.hex(3740.0)'0x1.d380000000000p+11'

5.5. Iterator Types¶

Python supports a concept of iteration over containers. This is implemented using two distinct methods; these are used to allow user-defined classes to support iteration. Sequences, described below in more detail, always support the iteration methods.

One method needs to be defined for container objects to provide iteration support:

()¶

Return an iterator object. The object is required to support the iterator protocol described below. If a container supports different types of iteration, additional methods can be provided to specifically request iterators for those iteration types. (An example of an object supporting multiple forms of iteration would be a tree structure which supports both breadth-first and depth-first traversal.) This method corresponds to the slot of the type structure for Python objects in the Python/C API.

The iterator objects themselves are required to support the following two methods, which together form the iterator protocol:

()¶

Return the iterator object itself. This is required to allow both containers and iterators to be used with the and statements. This method corresponds to the slot of the type structure for Python objects in the Python/C API.

()¶

Return the next item from the container. If there are no further items, raise the exception. This method corresponds to the slot of the type structure for Python objects in the Python/C API.

Python defines several iterator objects to support iteration over general and specific sequence types, dictionaries, and other more specialized forms. The specific types are not important beyond their implementation of the iterator protocol.

The intention of the protocol is that once an iterator’s method raises , it will continue to do so on subsequent calls. Implementations that do not obey this property are deemed broken. (This constraint was added in Python 2.3; in Python 2.2, various iterators are broken according to this rule.)

5.5.1. Generator Types¶

Python’s generators provide a convenient way to implement the iterator protocol. If a container object’s method is implemented as a generator, it will automatically return an iterator object (technically, a generator object) supplying the and methods. More information about generators can be found in the documentation for the yield expression.

5.6. Sequence Types — , , , , , , ¶

There are seven sequence types: strings, Unicode strings, lists, tuples, bytearrays, buffers, and xrange objects.

For other containers see the built in and classes, and the module.

String literals are written in single or double quotes: , . See String literals for more about string literals. Unicode strings are much like strings, but are specified in the syntax using a preceding character: , . In addition to the functionality described here, there are also string-specific methods described in the String Methods section. Lists are constructed with square brackets, separating items with commas: . Tuples are constructed by the comma operator (not within square brackets), with or without enclosing parentheses, but an empty tuple must have the enclosing parentheses, such as or . A single item tuple must have a trailing comma, such as .

Bytearray objects are created with the built-in function .

Buffer objects are not directly supported by Python syntax, but can be created by calling the built-in function . They don’t support concatenation or repetition.

Objects of type xrange are similar to buffers in that there is no specific syntax to create them, but they are created using the function. They don’t support slicing, concatenation or repetition, and using , , or on them is inefficient.

Most sequence types support the following operations. The and operations have the same priorities as the comparison operations. The and operations have the same priority as the corresponding numeric operations. [3] Additional methods are provided for Mutable Sequence Types.

This table lists the sequence operations sorted in ascending priority. In the table, s and t are sequences of the same type; n, i and j are integers:

OperationResultNotes
if an item of s is equal to x, else (1)
if an item of s is equal to x, else (1)
the concatenation of s and t(6)
equivalent to adding s to itself n times(2)
ith item of s, origin 0(3)
slice of s from i to j(3)(4)
slice of s from i to j with step k(3)(5)
length of s 
smallest item of s 
largest item of s 
index of the first occurrence of x in s 
total number of occurrences of x in s 

Sequence types also support comparisons. In particular, tuples and lists are compared lexicographically by comparing corresponding elements. This means that to compare equal, every element must compare equal and the two sequences must be of the same type and have the same length. (For full details see Comparisons in the language reference.)

Notes:

  1. When s is a string or Unicode string object the and operations act like a substring test. In Python versions before 2.3, x had to be a string of length 1. In Python 2.3 and beyond, x may be a string of any length.

  2. Values of n less than are treated as (which yields an empty sequence of the same type as s). Note that items in the sequence s are not copied; they are referenced multiple times. This often haunts new Python programmers; consider:

    What has happened is that is a one-element list containing an empty list, so all three elements of are references to this single empty list. Modifying any of the elements of modifies this single list. You can create a list of different lists this way:

    Further explanation is available in the FAQ entry How do I create a multidimensional list?.

    >>> lists=[[]]*3>>> lists[[], [], []]>>> lists[0].append(3)>>> lists[[3], [3], [3]]
    >>> lists=[[]foriinrange(3)]>>> lists[0].append(3)>>> lists[1].append(5)>>> lists[2].append(7)>>> lists[[3], [5], [7]]
  3. If i or j is negative, the index is relative to the end of sequence s: or is substituted. But note that is still .

  4. The slice of s from i to j is defined as the sequence of items with index k such that . If i or j is greater than , use . If i is omitted or , use . If j is omitted or , use . If i is greater than or equal to j, the slice is empty.

  5. The slice of s from i to j with step k is defined as the sequence of items with index such that . In other words, the indices are , , , and so on, stopping when j is reached (but never including j). When k is positive, i and j are reduced to if they are greater. When k is negative, i and j are reduced to if they are greater. If i or j are omitted or , they become “end” values (which end depends on the sign of k). Note, k cannot be zero. If k is , it is treated like .

  6. CPython implementation detail: If s and t are both strings, some Python implementations such as CPython can usually perform an in-place optimization for assignments of the form or . When applicable, this optimization makes quadratic run-time much less likely. This optimization is both version and implementation dependent. For performance sensitive code, it is preferable to use the method which assures consistent linear concatenation performance across versions and implementations.

    Changed in version 2.4: Formerly, string concatenation never occurred in-place.

5.6.1. String Methods¶

Below are listed the string methods which both 8-bit strings and Unicode objects support. Some of them are also available on objects.

In addition, Python’s strings support the sequence type methods described in the Sequence Types — str, unicode, list, tuple, bytearray, buffer, xrange section. To output formatted strings use template strings or the operator described in the String Formatting Operations section. Also, see the module for string functions based on regular expressions.

()¶

Return a copy of the string with its first character capitalized and the rest lowercased.

For 8-bit strings, this method is locale-dependent.

(width[, fillchar])¶

Return centered in a string of length width. Padding is done using the specified fillchar (default is a space).

Changed in version 2.4: Support for the fillchar argument.

(sub[, start[, end]])¶

Return the number of non-overlapping occurrences of substring sub in the range [start, end]. Optional arguments start and end are interpreted as in slice notation.

([encoding[, errors]])¶

Decodes the string using the codec registered for encoding. encoding defaults to the default string encoding. errors may be given to set a different error handling scheme. The default is , meaning that encoding errors raise . Other possible values are , and any other name registered via , see section Codec Base Classes.

Changed in version 2.3: Support for other error handling schemes added.

Changed in version 2.7: Support for keyword arguments added.

([encoding[, errors]])¶

Return an encoded version of the string. Default encoding is the current default string encoding. errors may be given to set a different error handling scheme. The default for errors is , meaning that encoding errors raise a . Other possible values are , , , and any other name registered via , see section Codec Base Classes. For a list of possible encodings, see section Standard Encodings.

Changed in version 2.3: Support for and and other error handling schemes added.

Changed in version 2.7: Support for keyword arguments added.

(suffix[, start[, end]])¶

Return if the string ends with the specified suffix, otherwise return . suffix can also be a tuple of suffixes to look for. With optional start, test beginning at that position. With optional end, stop comparing at that position.

Changed in version 2.5: Accept tuples as suffix.

([tabsize])¶

Return a copy of the string where all tab characters are replaced by one or more spaces, depending on the current column and the given tab size. Tab positions occur every tabsize characters (default is 8, giving tab positions at columns 0, 8, 16 and so on). To expand the string, the current column is set to zero and the string is examined character by character. If the character is a tab (), one or more space characters are inserted in the result until the current column is equal to the next tab position. (The tab character itself is not copied.) If the character is a newline () or return (), it is copied and the current column is reset to zero. Any other character is copied unchanged and the current column is incremented by one regardless of how the character is represented when printed.

>>> '01\t012\t0123\t01234'.expandtabs()'01 012 0123 01234'>>> '01\t012\t0123\t01234'.expandtabs(4)'01 012 0123 01234'
(sub[, start[, end]])¶

Return the lowest index in the string where substring sub is found within the slice . Optional arguments start and end are interpreted as in slice notation. Return if sub is not found.

Note

The method should be used only if you need to know the position of sub. To check if sub is a substring or not, use the operator:

>>> 'Py'in'Python'True
(*args, **kwargs

Perform a string formatting operation. The string on which this method is called can contain literal text or replacement fields delimited by braces . Each replacement field contains either the numeric index of a positional argument, or the name of a keyword argument. Returns a copy of the string where each replacement field is replaced with the string value of the corresponding argument.

See Format String Syntax for a description of the various formatting options that can be specified in format strings.

This method of string formatting is the new standard in Python 3, and should be preferred to the formatting described in String Formatting Operations in new code.

>>> "The sum of 1 + 2 is {0}".format(1+2)'The sum of 1 + 2 is 3'
(sub[, start[, end]])¶

Like , but raise when the substring is not found.

()¶

Return true if all characters in the string are alphanumeric and there is at least one character, false otherwise.

For 8-bit strings, this method is locale-dependent.

()¶

Return true if all characters in the string are alphabetic and there is at least one character, false otherwise.

For 8-bit strings, this method is locale-dependent.

()¶

Return true if all characters in the string are digits and there is at least one character, false otherwise.

For 8-bit strings, this method is locale-dependent.

()¶

Return true if all cased characters [4] in the string are lowercase and there is at least one cased character, false otherwise.

For 8-bit strings, this method is locale-dependent.

()¶

Return true if there are only whitespace characters in the string and there is at least one character, false otherwise.

For 8-bit strings, this method is locale-dependent.

()¶

Return true if the string is a titlecased string and there is at least one character, for example uppercase characters may only follow uncased characters and lowercase characters only cased ones. Return false otherwise.

For 8-bit strings, this method is locale-dependent.

()¶

Return true if all cased characters [4] in the string are uppercase and there is at least one cased character, false otherwise.

For 8-bit strings, this method is locale-dependent.

(iterable

Return a string which is the concatenation of the strings in iterable. If there is any Unicode object in iterable, return a Unicode instead. A will be raised if there are any non-string or non Unicode object values in iterable. The separator between elements is the string providing this method.

(width[, fillchar])¶

Return the string left justified in a string of length width. Padding is done using the specified fillchar (default is a space). The original string is returned if width is less than or equal to .

Changed in version 2.4: Support for the fillchar argument.

()¶

Return a copy of the string with all the cased characters [4] converted to lowercase.

For 8-bit strings, this method is locale-dependent.

([chars])¶

Return a copy of the string with leading characters removed. The chars argument is a string specifying the set of characters to be removed. If omitted or , the chars argument defaults to removing whitespace. The chars argument is not a prefix; rather, all combinations of its values are stripped:

Changed in version 2.2.2: Support for the chars argument.

>>> ' spacious '.lstrip()'spacious '>>> 'www.example.com'.lstrip('cmowz.')'example.com'
(sep

Split the string at the first occurrence of sep, and return a 3-tuple containing the part before the separator, the separator itself, and the part after the separator. If the separator is not found, return a 3-tuple containing the string itself, followed by two empty strings.

(old, new[, count])¶

Return a copy of the string with all occurrences of substring old replaced by new. If the optional argument count is given, only the first count occurrences are replaced.

(sub[, start[, end]])¶

Return the highest index in the string where substring sub is found, such that sub is contained within . Optional arguments start and end are interpreted as in slice notation. Return on failure.

(sub[, start[, end]])¶

Like but raises when the substring sub is not found.

(width[, fillchar])¶

Return the string right justified in a string of length width. Padding is done using the specified fillchar (default is a space). The original string is returned if width is less than or equal to .

Changed in version 2.4: Support for the fillchar argument.

(sep

Split the string at the last occurrence of sep, and return a 3-tuple containing the part before the separator, the separator itself, and the part after the separator. If the separator is not found, return a 3-tuple containing two empty strings, followed by the string itself.

([sep[, maxsplit]])¶

Return a list of the words in the string, using sep as the delimiter string. If maxsplit is given, at most maxsplit splits are done, the rightmost ones. If sep is not specified or , any whitespace string is a separator. Except for splitting from the right, behaves like which is described in detail below.

([chars])¶

Return a copy of the string with trailing characters removed. The chars argument is a string specifying the set of characters to be removed. If omitted or , the chars argument defaults to removing whitespace. The chars argument is not a suffix; rather, all combinations of its values are stripped:

Changed in version 2.2.2: Support for the chars argument.

>>> ' spacious '.rstrip()' spacious'>>> 'mississippi'.rstrip('ipz')'mississ'
([sep[, maxsplit]])¶

Return a list of the words in the string, using sep as the delimiter string. If maxsplit is given, at most maxsplit splits are done (thus, the list will have at most elements). If maxsplit is not specified or , then there is no limit on the number of splits (all possible splits are made).

If sep is given, consecutive delimiters are not grouped together and are deemed to delimit empty strings (for example, returns ). The sep argument may consist of multiple characters (for example, returns ). Splitting an empty string with a specified separator returns .

If sep is not specified or is , a different splitting algorithm is applied: runs of consecutive whitespace are regarded as a single separator, and the result will contain no empty strings at the start or end if the string has leading or trailing whitespace. Consequently, splitting an empty string or a string consisting of just whitespace with a separator returns .

For example, returns , and returns .

(

0 thoughts on “C++ String Class Assignment Operator Python

Leave a Reply

Your email address will not be published. Required fields are marked *