Hive partitioning not working with dynamic variable

If I run

set hivevar:a = 1;
select * from t1 where partition_variable=${a};

Hive only pulls in the records from the appropriate partition. Alternately if I run

set hivevar:b = 6;
set hivevar:c = 5;
set hivevar:a = ${b}-${c};
select * from t1 where partition_variable=${a};

The condition on partition_variable is treated as a predicate rather than a partition, and hive goes through all records in the table.

This is obviously a contrived example, but in my particular use case it is necessary. Is there anyway to force hive to use this for partitioning?

Thanks in advance.

Answers


Is the partition variable the column on which partition occurs. It works with following.

create table newpart
(productOfMonth string)
partitioned by (month int);


hive> select * from newpart;
OK
Cantaloupes 10 
Pumpkin     11

set hivevar:lastmonth = 11;
set hivevar:const = 1;
set hivevar:prevmonth = ${lastmonth}-${const};

hive> select * from newpart
    > where month = ${prevmonth};
OK
Cantaloupes 10

I was never able to get partitioning to work properly with dynamically generated hive variables, but a simple workaround was to create a table containing the variables and join on them rather than using them in the where clause.


Need Your Help

Framework Vs. API

api frameworks

Now, this may be a silly question but sometimes the terms Framework and API are used interchangeably. The way I see it is that a Framework is a bigger more generic thing, containing many API's, that

Grunt - Queueing a task to run after a previous task completes

javascript css gruntjs minify task-queue

I looked at the official Grunt documentation for creating tasks here