How to make my custom type to work with "range-based for loops"?

Like many people these days I've been trying the different features that C+11 brings. One of my favorites is the "range-based for loops".

I understand that:

for(Type& v : a) { ... }

Is equivalent to:

for(auto iv = begin(a); iv != end(a); ++iv)
{
  Type& v = *iv;
  ...
}

And that begin() simply returns a.begin() for standard containers.

But what if I want to make my custom type "range-based for loop"-aware ?

Should I just specialize begin() and end() ?

If my custom type belongs to the namespace xml, should I define xml::begin() or std::begin() ?

In short, what are the guidelines to do that ?

Answers


The standard has been changed since the question (and most answers) were posted in the resolution of this defect report.

The way to make a for(:) loop work on your type X is now one of two ways:

  • Create member X::begin() and X::end() that return something that acts like an iterator

  • Create a free function begin(X&) and end(X&) that return something that acts like an iterator, in the same namespace as your type X.¹

And similar for const variations. This will work both on compilers that implement the defect report changes, and compilers that do not.

The objects returned do not have to actually be iterators. The for(:) loop, unlike most parts of the C++ standard, is specified to expand to something equivalent to:

for( range_declaration : range_expression )

becomes:

{
  auto && __range = range_expression ;
  for (auto __begin = begin_expr,
            __end = end_expr;
            __begin != __end; ++__begin) {
    range_declaration = *__begin;
    loop_statement
  }
}

where the variables beginning with __ are for exposition only, and begin_expr and end_expr is the magic that calls begin/end.²

The requirements on the begin/end return value are simple: You must overload pre-++, ensure the initialization expressions are valid, binary != that can be used in a boolean context, unary * that returns something you can assign-initialize range_declaration with, and expose a public destructor.

Doing so in a way that isn't compatible with an iterator is probably a bad idea, as future iterations of C++ might be relatively cavalier about breaking your code if you do.

As an aside, it is reasonably likely that a future revision of the standard will permit end_expr to return a different type than begin_expr. This is useful in that it permits "lazy-end" evaluation (like detecting null-termination) that is easy to optimize to be as efficient as a hand-written C loop, and other similar advantages.


¹ Note that for(:) loops store any temporary in an auto&& variable, and pass it to you as an lvalue. You cannot detect if you are iterating over a temporary (or other rvalue); such an overload will not be called by a for(:) loop. See [stmt.ranged] 1.2-1.3 from n4527.

² Either call the begin/end method, or ADL-only lookup of free function begin/end, or magic for C-style array support. Note that std::begin is not called unless range_expression returns an object of type in namespace std or dependent on same.


In c++17 the range-for expression has been updated

{
  auto && __range = range_expression ;
  auto __begin = begin_expr;
  auto __end = end_expr
  for (;__begin != __end; ++__begin) {
    range_declaration = *__begin;
    loop_statement
  }
}

with the types of __begin and __end have been decoupled.

This permits the end iterator to not be the same type as begin. Your end iterator type can be a "sentinel" which only supports != with the begin iterator type.

A practical example of why this is useful is that your end iterator can read "check your char* to see if it points to '0'" when == with a char*. This allows a C++ range-for expression to generate optimal code when iterating over a null-terminated char* buffer.

struct null_sentinal_t {
  template<class Rhs,
    std::enable_if_t<!std::is_same<Rhs, null_sentinal_t>{},int> =0
  >
  friend bool operator==(Rhs const& ptr, null_sentinal_t) {
    return !*ptr;
  }
  template<class Rhs,
    std::enable_if_t<!std::is_same<Rhs, null_sentinal_t>{},int> =0
  >
  friend bool operator!=(Rhs const& ptr, null_sentinal_t) {
    return !(ptr==null_sentinal_t{});
  }
  template<class Lhs,
    std::enable_if_t<!std::is_same<Lhs, null_sentinal_t>{},int> =0
  >
  friend bool operator==(null_sentinal_t, Lhs const& ptr) {
    return !*ptr;
  }
  template<class Lhs,
    std::enable_if_t<!std::is_same<Lhs, null_sentinal_t>{},int> =0
  >
  friend bool operator!=(null_sentinal_t, Lhs const& ptr) {
    return !(null_sentinal_t{}==ptr);
  }
  friend bool operator==(null_sentinal_t, null_sentinal_t) {
    return true;
  }
  friend bool operator!=(null_sentinal_t, null_sentinal_t) {
    return false;
  }
};

live example in a compiler without full C++17 support; for loop manually expanded.


The relevant part of the standard is 6.5.4/1:

if _RangeT is a class type, the unqualified-ids begin and end are looked up in the scope of class _RangeT as if by class member access lookup (3.4.5), and if either (or both) finds at least one declaration, begin- expr and end-expr are __range.begin() and __range.end(), respectively;

— otherwise, begin-expr and end-expr are begin(__range) and end(__range), respectively, where begin and end are looked up with argument-dependent lookup (3.4.2). For the purposes of this name lookup, namespace std is an associated namespace.

So, you can do any of the following:

  • define begin and end member functions
  • define begin and end free functions that will be found by ADL (simplified version: put them in the same namespace as the class)
  • specialize std::begin and std::end

std::begin calls the begin() member function anyway, so if you only implement one of the above, then the results should be the same no matter which one you choose. That's the same results for ranged-based for loops, and also the same result for mere mortal code that doesn't have its own magical name resolution rules so just does using std::begin; followed by an unqualified call to begin(a).

If you implement the member functions and the ADL functions, though, then range-based for loops should call the member functions, whereas mere mortals will call the ADL functions. Best make sure they do the same thing in that case!

If the thing you're writing implements the container interface, then it will have begin() and end() member functions already, which should be sufficient. If it's a range that isn't a container (which would be a good idea if it's immutable or if you don't know the size up front), you're free to choose.

Of the options you lay out, note that you must not overload std::begin(). You are permitted to specialize standard templates for a user-defined type, but aside from that, adding definitions to namespace std is undefined behavior. But anyway, specializing standard functions is a poor choice if only because the lack of partial function specialization means you can only do it for a single class, not for a class template.


I write my answer because some people might be more happy with simple real life example without STL includes.

I have my own plain only data array implementation for some reason, and I wanted to use the range based for loop. Here is my solution:

 template <typename DataType>
 class PodArray {
 public:
   class iterator {
   public:
     iterator(DataType * ptr): ptr(ptr){}
     iterator operator++() { ++ptr; return *this; }
     bool operator!=(const iterator & other) const { return ptr != other.ptr; }
     const DataType& operator*() const { return *ptr; }
   private:
     DataType* ptr;
   };
 private:
   unsigned len;
   DataType *val;
 public:
   iterator begin() const { return iterator(val); }
   iterator end() const { return iterator(val + len); }

   // rest of the container definition not related to the question ...
 };

Then the usage example:

PodArray<char> array;
// fill up array in some way
for(auto& c : array)
  printf("char: %c\n", c);

Should I just specialize begin() and end() ?

As far as I know, that is enough. You also have to make sure that incrementing the pointer would get from the begin to the end.

Next example (it is missing const version of begin and end) compiles and works fine.

#include <iostream>
#include <algorithm>

int i=0;

struct A
{
    A()
    {
        std::generate(&v[0], &v[10], [&i](){  return ++i;} );
    }
    int * begin()
    {
        return &v[0];
    }
    int * end()
    {
        return &v[10];
    }

    int v[10];
};

int main()
{
    A a;
    for( auto it : a )
    {
        std::cout << it << std::endl;
    }
}

Here is another example with begin/end as functions. They have to be in the same namespace as the class, because of ADL :

#include <iostream>
#include <algorithm>


namespace foo{
int i=0;

struct A
{
    A()
    {
        std::generate(&v[0], &v[10], [&i](){  return ++i;} );
    }

    int v[10];
};

int *begin( A &v )
{
    return &v.v[0];
}
int *end( A &v )
{
    return &v.v[10];
}
} // namespace foo

int main()
{
    foo::A a;
    for( auto it : a )
    {
        std::cout << it << std::endl;
    }
}

In case you want to back a class's iteration directly with its std::vector or std::map member, here is the code for that:

#include <iostream>
using std::cout;
using std::endl;
#include <string>
using std::string;
#include <vector>
using std::vector;
#include <map>
using std::map;


/////////////////////////////////////////////////////
/// classes
/////////////////////////////////////////////////////

class VectorValues {
private:
    vector<int> v = vector<int>(10);

public:
    vector<int>::iterator begin(){
        return v.begin();
    }
    vector<int>::iterator end(){
        return v.end();
    }
    vector<int>::const_iterator begin() const {
        return v.begin();
    }
    vector<int>::const_iterator end() const {
        return v.end();
    }
};

class MapValues {
private:
    map<string,int> v;

public:
    map<string,int>::iterator begin(){
        return v.begin();
    }
    map<string,int>::iterator end(){
        return v.end();
    }
    map<string,int>::const_iterator begin() const {
        return v.begin();
    }
    map<string,int>::const_iterator end() const {
        return v.end();
    }

    const int& operator[](string key) const {
        return v.at(key);
    }
    int& operator[](string key) {
        return v[key];
    } 
};


/////////////////////////////////////////////////////
/// main
/////////////////////////////////////////////////////

int main() {
    // VectorValues
    VectorValues items;
    int i = 0;
    for(int& item : items) {
        item = i;
        i++;
    }
    for(int& item : items)
        cout << item << " ";
    cout << endl << endl;

    // MapValues
    MapValues m;
    m["a"] = 1;
    m["b"] = 2;
    m["c"] = 3;
    for(auto pair: m)
        cout << pair.first << " " << pair.second << endl;
}

Here, I am sharing the simplest example of creating custom type, that will work with "range-based for loop":

#include<iostream>
using namespace std;

template<typename T, int sizeOfArray>
class MyCustomType
{
private:
    T *data;
    int indx;
public:
    MyCustomType(){
        data = new T[sizeOfArray];
        indx = -1;
    }
    ~MyCustomType(){
        delete []data;
    }
    void addData(T newVal){
        data[++indx] = newVal;
    }

    //write definition for begin() and end()
    //these two method will be used for "ranged based loop idiom"
    T* begin(){
        return &data[0];
    }
    T* end(){
        return  &data[sizeOfArray];
    }
};
int main()
{
    MyCustomType<double, 2> numberList;
    numberList.addData(20.25);
    numberList.addData(50.12);
    for(auto val: numberList){
        cout<<val<<endl;
    }
    return 0;
}

Hope, it will be helpful for some novice developer like me :p :) Thank You.


Chris Redford's answer also works for Qt containers (of course). Here is an adaption (notice I return a constBegin(), respectively constEnd() from the const_iterator methods):

class MyCustomClass{
    QList<MyCustomDatatype> data_;
public:    
    // ctors,dtor, methods here...

    QList<MyCustomDatatype>::iterator begin() { return data_.begin(); }
    QList<MyCustomDatatype>::iterator end() { return data_.end(); }
    QList<MyCustomDatatype>::const_iterator begin() const{ return data_.constBegin(); }
    QList<MyCustomDatatype>::const_iterator end() const{ return data_.constEnd(); }
};

Need Your Help

Outer glow effect to border

wpf border glow bitmapeffect

How to provide the outer glow effect to border?

config.assets.compile=true in Rails production, why not?

ruby-on-rails asset-pipeline production-environment

The default Rails app installed by rails new has config.assets.compile = false in production.