Using character as relational operator in C/C++

[Wybiral]

Can+~, thanks for that post - you've cleared up quite a few things for me. I think I understand what Wybiral and you are saying. By using that method, Wybiral's trying to remove the staticness of the implementation by defining blocks that can be "pick and mixed" so to speak.

It's not just a matter of dynamicness, it's also about less rewriting of the same crap. When you write everything out procedurally, you often end up reimplementing the same things because they aren't as easy to interchange... Which leads to lots of copy&paste code and bloat. Streams allow you to abstract away some of your reoccurring patterns.

For example, if I understood Wybiral's code, it generates all the needed Fibonacci numbers, testing them (and dismissing some), and then squaring them.

If I were to use "map(add1, filter(is_even, fibs()))" what's really going on is this...

map says "pull a value from filter and do add1(i)"
filter says "pull a value from fibs() and return it if is_even(i)"
fibs simple spits out values when asked

This is repeated until "map" isn't able to pull a value, in this case, it's infinite, but if I said:

take(20, map(add1, filter(is_even, fibs())))

Then the "take" is what starts the "pulling" and it will stop after pulling 20 values through. It doesn't generate all of the fibonacci numbers necessary, it only pulls them out when needed as it goes. If you would be more satisfied, you could write a even_fibs generator instead of doing the filter so you can use your optimization... But the rest is exactly the same thing that would happen in an iteration, just more modular.

My method on the other hand uses the fact that every third Fibonacci number is even. With this knowledge I need not even check which numbers are even - they all are!

This is under the assumption that you can't just write an "every_third_fib" generator You can use those optimizations when needed, but piece it all together as a stream of generators.

For other cases, I would expect taking Wybiral's approach could even lead to a far less efficient algorithm (computational complexity wise).

Not necessarily...
In this case, with just streams, so it's not as interesting, it's just another way of abstractly handling the iterations. But the map/filter/reduce concept is VERY powerful when it comes to reducing complexity and especially paralleling the execution.

When I do "map(f1, filter(f2, items))" there's nothing holding this back from doing the filter test on each item in parallel, likewise with the map (after the filtered objects have been collected). Scalability then starts to come naturally. This is a powerful edge over constantly thinking in terms of iteration (where you're explicitly expressing the problem in a sequential nature), I don't care what order they happen in... Do it all at once, then collect them... Do them backwards... Whatever...

Not everything needs to be dynamic all the time. But when they do, you could just use "if"s with several different functions. There are quite easy ways to achieve dynamics like that.

Then you end up having to code those parts into the loops, and doing the conditions each loop. It's not as easy to write modularly where you can interchange pieces and reuse everything. It's not just about runtime dynamicness, it's about reusable, modular, scalable ways of dealing with large sets of data.

[Lster]

This is under the assumption that you can't just write an "every_third_fib" generator You can use those optimizations when needed, but piece it all together as a stream of generators.

But then you would, effectively, be using my method... and perhaps adding streams on the end, which in this case would be useless...

Not necessarily...
In this case, with just streams, so it's not as interesting, it's just another way of abstractly handling the iterations. But the map/filter/reduce concept is VERY powerful when it comes to reducing complexity and especially paralleling the execution.

How can they reduce complexity? I would love to see an example.

When I do "map(f1, filter(f2, items))" there's nothing holding this back from doing the filter test on each item in parallel, likewise with the map (after the filtered objects have been collected). Scalability then starts to come naturally. This is a powerful edge over constantly thinking in terms of iteration (where you're explicitly expressing the problem in a sequential nature), I don't care what order they happen in... Do it all at once, then collect them... Do them backwards... Whatever...

That's really looking to the future. I doubt many people have lots of cores on their computers. However it's an interesting point considering multicores look like they're here to stay. It definitely isn't impossible in C with multithreading - or hard for that matter.

In fact, I've been doing a lot recently with multithreading. My game engine, for example, is fully written in C and can support any number of processors (theoretically). For that I have a tasking system. A stack of tasks are stored and each one has dependencies. Each thread keeps checking the stack to see if a task is accessible; when one is, it runs it marking it done.

[Wybiral]

But then you would, effectively, be using my method... and perhaps adding streams on the end, which in this case would be useless...

Yes, Lster, in this EXACT, simplified, one-time-only case you can argue that none of this matters. But think bigger than this... Once you write that generator, you can use it anywhere you want! You can iterate over it with a for loop... You can pass it through filters, maps, reduces... You can take the nth element, the first n elements... Etc. It's modular. Reusable. Simple.

You keep referring to this exact example, stop. I'm talking about an entire concept, I don't care how easy it is to write this problem in C, the CONCEPT is what's important. Not this simple example.

How can they reduce complexity? I would love to see an example.

OK, perhaps I worded that incorrectly. Using streams, for instance, can increase the memory efficiency by not having to rebuild lists over-and-over (think about the spacial efficiency). And, using a parallel map/reduce you can scale the algorithms to as many processors or servers as you'd like. I know... You can do these things procedurally. It's just harder and more work that way. Spend the rest of your life writing millions of "for" loops if you want... But there ARE better ways.

This conversation has broken into two topics now, streams and functional map/reduce. I want to make it clear that we're talking about two things now. Streams go beyond map/reduce, you can do lots of things with them. Map/reduce is just a common use for them, and it's also an elegant and simple way to massively parallel various algorithms.

That's really looking to the future. I doubt many people have lots of cores on their computers. However it's an interesting point considering multicores look like they're here to stay. It definitely isn't impossible in C with multithreading - or hard for that matter.

WRT map/filter/reduce (not streams anymore):
This goes FAR beyond simple multithreading... This scales to entire server farms with little effort on the user. Google, for instance, has employed mapreduce for many of their applications. It's simply the most natural way to deal with large sets of data. Constantly thinking sequentially in "for" loops just isn't going to cut it, it's a broken concept when you're trying to parallel your code, in its essence it is explicitly saying "I'm sequential".

Map and filter are much different... The order of execution doesn't matter, and that's the beauty of it, and functional programming in-general!

[Lster]

Yes, Lster, in this EXACT, simplified, one-time-only case you can argue that none of this matters. But think bigger than this... Once you write that generator, you can use it anywhere you want! You can iterate over it with a for loop... You can pass it through filters, maps, reduces... You can take the nth element, the first n elements... Etc. It's modular. Reusable. Simple.

You keep referring to this exact example, stop. I'm talking about an entire concept, I don't care how easy it is to write this problem in C, the CONCEPT is what's important. Not this simple example.

I get your point but I don't think the situation you describe arises much. I guess all of this comes down to mindset. I have never had a problem in which I would use your techniques over my own. I can always find a way around the solution and usually very efficiently without using such constructs. I guess it really does come down to ease.

Really, I doubt there are any problems you can give me which require those kinds of methods. Or even any where I would have to go through a very long winded method.

Using streams, for instance, can increase the memory efficiency by not having to rebuild lists over-and-over (think about the spacial efficiency).

Actually, my proposed way would not need to store a list.

[Wybiral]

I get your point but I don't think the situation you describe arises much. I guess all of this comes down to mindset. I have never had a problem in which I would use your techniques over my own. I can always find a way around the solution and usually very efficiently without using such constructs. I guess it really does come down to ease.

Of course, because you've never used it before

As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.

"Beating the Average", Paul Graham

[Lster]

Yeah, but I get along fine. I never find the need for Python, but it's often desirable.

It's not that I'm closed to new languages, it's not that I don't try to explore them, it's just I don't see the need. The term "blub" programmer is really quite useless - by its loose definition almost every programmer is one. It would also appear Paul Graham is talking more about the business side of programming which is by no means the whole of it, nor the part I care for.

[CptPicard]

As I said before, functional-programming concepts can reduce "complexity" simply by revealing useful problem structure, which yields a better algorithm. A lot of stuff is just very naturally expressed functionally, simply because mathematics tends to already provide a good description. Imperative programming has the added problem of actually describing the "how", while functionally, you just tell the computer "what". And such abstract descriptions tend to be very very general and can be very elegant.

Pure-functional programming's ability to scale and to have all sorts of other fancy transformations applied to it shouldn't be underestimated either, parallelization is the future... and you should also remember that parallelization has a close cousin, distributed computing, which derives similar benefits.

You can get hidden complexity from things like generating your entire list in RAM etc... but you're not supposed to be stupid using any programming paradigm...

It's not that I'm closed to new languages, it's not that I don't try to explore them, it's just I don't see the need. The term "blub" programmer is really quite useless - by its loose definition almost every programmer is one.

Everyone is in some ways always a beginner... but blubness is a whole mindset of its own really. For example I have done enough to never underestimate some concept that I don't "feel the need for" because it tends to usually just mean that I haven't looked into it enough.

It would also appear Paul Graham is talking more about the business side of programming which is by no means the whole of it, nor the part I care for.

Actually, he is pointing out that "surprisingly" Lisp is a business-capable language too. Lisp was supposedly the Holy Grail of AI back in the day in the 1960s... it wasn't, of course, but it got a lot, lot of stuff right.

Your last sentence really is the core reason why you should look into all of this. Just because you theoretically can do it all in C, doesn't mean there aren't extremely insightful and cool things out there in higher-level languages. You're starting to sound like Shining Arcanine ... read SICP and then come tell Wybiral and I that you don't need to know Scheme because you're interested in the non-business-side of programming

[Bichromat]

As I said before, functional-programming concepts can reduce "complexity" simply by revealing useful problem structure, which yields a better algorithm. A lot of stuff is just very naturally expressed functionally, simply because mathematics tends to already provide a good description. Imperative programming has the added problem of actually describing the "how", while functionally, you just tell the computer "what". And such abstract descriptions tend to be very very general and can be very elegant.

But it can be very hard to wrap your mind around these concepts. I'm trying to learn Haskell and I just don't see how I could do e.g. a physics simulation, because a loop over a certain number of time steps just seems the "right" way to do it... Sometimes, the "how" is the natural way to describe a problem.

[CptPicard]

But it can be very hard to wrap your mind around these concepts. I'm trying to learn Haskell and I just don't see how I could do e.g. a physics simulation, because a loop over a certain number of time steps just seems the "right" way to do it...

True... well, pure-functional languages like Haskell are a bit strange in that sense. That's why I am a lisper and not a haskeller, lisp lets me have my mutable state when I must (although Clojure is trying hard to get rid of it).

I never understood how monads work, but I suppose one day I will... and usually when I have understood stuff like that, it really is a revelation that gives me ideas not only for that specific language, but other languages as well.

But, of course, I am not advocating functional programming for everything

How you'd model that particular issue typically is btw an infinite list of states and associated "time-indexes". Then you just need to have some kind of state-transformation function that gets a state and returns a new state. Monads (somehow) encapsulate this sort of stuff. But yes, I have always felt like Haskell is too computer-sciencey to be really practicable for other scientists as a coding language.

[pmasiar]

My language is better than yours... bla bla bla

I think that discussion about generators in Python nicely underscored the point we were trying to make: speed of execution is not the only measure to judge a language, and Python has different ways to conquer complexity - which you, as a blub programmer, cannot see, and do not consider when judging the language. Which is your right of course.

To answer your question: No, Google does not run queries in Python, they are in C++, and some less time-critical are in Java. But majority of Google code is not to run queries, but to manage deployment of hundreds of thousands of servers - and that is written in Python. As Alex Martelli (head of production/deployment and one of top Python gurus) said: "we program in Python if we can get away with it, in C++ if we must".

Python is not the best tool for every job under the sun - but neither is C++. But Python **is** best tool for about 80% of the jobs, that's why it is always my first choice.